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EXECUTIVE SUMMARY 



In recent decades, there has been a marked rise in the concern with performance 
management in both the public and the private -sector. The evolution of performance 
management in the public sector has been neither steady or consistent. However, more 
recendy the need to stress program management to produce the desired outcomes has been 
felt more consistently across the board. A number of key cross-cutting design and 
implementation issues form the basis for this discussion of program management concerns. 
These are: Legislative Purpose; Level of Use; the Role of Performance Standards; 
Appropriate Measures of Performance; Target Groups; and General Implementation Issues. 

Program performance standards, incentives and sanctions in these programs are 
discussed from several perspectives, with attention given to some of the more salient 
features of giant-in-aid programs and to important differences between public and private 
sector management environments. Cross-cutting issues are emphasized because of the 
perceived commonality of their missions: all are grant-in-aid programs funded primarily by 
federal and state governments for the development or more complete utilization of human 
resources. 

THE PROGRAMS AND THEIR MANAGEMENT SYSTEMS 

All of the programs described here include among their major goals job placement 
in the relatively near- term and self-sufficiency over the longer-term. These programs 
include: job training as embodied by the federal/state/local system established under the 
Job Training Partnership Act of 1982; job placement or the Employment Service authorized 
by the Wagner- Peyser Act of 1933; postsecondary vocational education or Voc Ed, recently 
reauthorized under the Carl D. Perkins Vocational Education Act of 1984; and a number of 
welfare-to-work programs, ranging from the Work Incentive (WIN) Program for recipients 
of federal/state Aid to Families with Dependent Children (AFDC) to the Food Stamp 
Employment and Training Program. Brief attention is also given to current proposals for 
welfare reform. 

Job Training. JTPA provides training and employment for economically 
disadvantaged adults and youth under Title IIA and dislocated workers under Title III. 
Summer jobs and training are offered to disadvantaged youth under Title IIB. Performance 
standards including incentives and sanctions now play a prominent role in training 
programs for adults and youth operated by PICs under Title IIA. Their intended role is 
more ambiguous in Title III. The performance management system under Title IIA has 
been developed incrementally since 1983 with intensive input from state/local entities and 
public interest groups. It is a hierarchical system, with roles and responsibilities for 
federal, state and local actors differentiated in rough relation to their distance from service 
provision. In keeping with legislative intent, the system is largely performance-driven, 
emphasizing outcomes — both at-termination and post-program — over in-program 
measures. There are both positive rewards and negative sanctions with governors now 
playing a key role in performance mangement. 
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Job Placement/Labor Exchange. State Employment Service Agencies operate 
as a federal/state partnership, serving both employers and potential employees, facilitating 
the labor exchange. Eligibility for services is open for both employers and employees, 
although some workers (e.g., migrant/seasonal farmworkers, handicapped and veterans) 
are given special priority. States now receive most of their funding based on need, as 
measured by relative shares of unemployment and the civilian labor force. The federal 
allocation flows directly to governors rather than to SESAs, with funds divided into two 
components: 90 percent for the provision of basic labor exchange services locally, through 
a substate funding allocation determined by the governor; .nd the remaining 10 percent 
reserved at the governor's discretion for performance incentives, services for groups with 
special needs, and the added costs of exemplary models. Concerns with performance 
management and the use of standards and incentives have been on the increase in the labor 
exchange system. Responsibility for ES program management, including the quality of the 
outcomes, rests with the governor who has the authority to foster coordination between the 
job training and job placement systems within the state, including establishing state-specific 
performance standards and accompanying incentive systems. The Secretary of Labor may 
set national ES standards. State and substate models designed to establish and adjust 
performance standards have been developed by the Secretary.for state and local programs. 

Postsecondary Vccationai Education, Vocational education can only loosely 
be described as a "system." Secondary and postsecondary training is offered by thousands 
of providers, including both public and private institutions, high schools, technical 
institutes, community colleges, four-year secondary institutions, correctional facilities, and 
proprietary schools. There is wide variation in the scope, nature and quality of these 
programs. The Perkins Act stresses two primary objectives: improved program quality 
and access. It promotes a highly categorized approach to Voc Ed programming with 
setasides for key groups and programs. Federal dollars account for only about 6 of every 
100 dollars spent on postsecondary programs. The secondary/postsecondary funding split 
is a state-level decision resting with the state education board; not surprisingly, the share 
allotted to postsecondary Voc Ed varies widely from state to state. 
Voc Ed has been inordinately process-oriented. The Perkins Act does not build in 
standards or add substantively to the performance orientation of postsecondary Voc Ed. 
States are left to their own devices for postsecondary Voc Ed performance management. 

Welfare-to-Work Programs for AFDC Recipients. Since 1981, there have 
been substantial changes in WIN, especially reduced funding and increased state discretion, 
giving rise to considerable within- and between-state program variation. Nationally, 90 
percent of WIN*s funding is federal. WIN has never received sufficient funding to do 
much more than comply with program "registration" for most participants. State discretion 
is high, but structured, regular documentation and reporting regarding the use of this 
discretion remains low. Despite considerable attention given to evaluations of 
work/welfare program interventions for AFDC recipients, translation into operational 
performance standards, incentives and sanctions has been lacking. More recently, 
researchers, policymakers and program administrators have begun to focus on the need for 
standards in the management of these efforts, particularly in the on-going debate over 
welfare reform. 




Food Stamp Employment and Training Programs. The nev/ program for 
Food Stamp recipients, administered by the U.S. Departm^.nt of Agriculture, is to be 
designed and operated by the state welfare agency. States have wide discretion in how 
these programs are to be run, including the activities. and services to be provided and the 
choice of service providers. Programs now have participation standards. Outcome 
atajidards have been discussed but not implemented. Because of limited federal funding 
little substantive intervention cai'i realistically be expected. 

SYNTHESIS OF THE ISSUES 

A number of major and minor issues should be considered by national and state 
policymakers. The following synthesis focuses on several sets of issues: broad contextual 
issues; major cross-cutting design and implementation issues; and several minor issues. 

Contextual Issues 

Performance management systems, and the standards and incentives which are an 
integral component of these systems, should be designed with their operational and 
environmental context in mind. Resolving these contextual issues may be a pre-condition 
for designing effective management systems for human resource programs. 

Grant-in-Aid Programs. All of the programs examined here are federal grant- 
in-aid programs. Although the extent to which they are funded from federal, state or local 
dollars differs widely, each is a grant-in-aid program with a substantial share of federal 
resources involved. Only postsecondary vocational education is dominated by state (and 
local) funding. The programs' grant-in-aid character has important implications for 
management systems and measures. The greater the share of federal funds, the greater the 
responsibility and the more federal strings can be attached to state and local programs. 
Programs with integrated state/local program delivery systems, such as ES, WIN and Food 
Stamps, have greater potential for state/local control; JTPA and Voc Ed must rely more 
upon influence due to the relative autonomy of their local delivery systems. The extent to 
which management systems with effective standards can be mandated and implemented 
from the federal down to the local level depends on the share of federal funding involved 
and the degree of influence which can be cultivated. 

Enhanced State Discretion. The trend in the 1980s has been towards an 
enhanced . j role in planning and operational decisionmaking for human resource 
development. This is especially evident in JTPA, ES and WIN. While the Perkins Act 
offers a counterpoint to this trend, states still retain enormous discretion in Voc Ed when all 
non-federal resources are considered. Governors have been vested with the responsibility, 
the authority and the funding to make critical decisions regarding human resources, and, 
increasingly, this extends to performance management functions. Two major constraints 
face the governors in managing these programs effectively: first, as elected officials, they 
may not be very well suited for it; and second, the tool kits available to them may have been 
too tightly circumscribed. 




Market Bias, As performance management has been introduced into human 
resource programs, there has been a tendency to search for market-based principles for 
guidance. Emphasis on "bottom-line" management approaches in JTPA, ES, Food Stamps 
and now welfare reform has seemingly become an obsession. Accountability or results- 
oriented management in the use of public funds must now be part of any and all programs. 
However, these are the very people and programs for which the market has apparently 
failed. To expect pure private sector approaches to adapt well is unreasonable. Before 
similar approaches are extended to Voc Ed or welfare-to-work programs, this apparent 
market bias merits careful scrutiny. If private sector management principles and 
expectations are going to bf^ applied successfully to publicly-funded human resource 
programs, modifications must be made. 

Major Issues 

Ambiguous Goals, The mission of many of these programs (e.g., JTPA and 
WIN) is moving individuals and their families towards self-sufficiency. But, there is 
enormous ambiguity in the actual goals of these efforts. The worst offender in this regard 
is Voc Ed which appears to be a program for all reasons: Voc Ed exists to provide 
vocational and non-vocational training for "students" who plan to be botli in and out of the 
labor market. It is all but impossible to hold such a system accountable or to design an 
effective management system when its goals and objectives cannot be clearly articulated. 

Most of these programs have relatively clear missions and may need minor 
clarification of goals and objectives. JTPA Title IIA has gone the farthest in this regard, 
laying out a clear statement of purpose and suggesting measures, including immediate 
termination-based measures and longer-term postprogram ones, tied to program objectives 
and differentiated at least by adult and youth groups. It is hard to imagine clearer language 
than that found in JTPA. ES could use improved definition, and the welfare-related 
programs, as suggested by the reform proposals, appear to be headed in the right direction. 
Voc Ed alone seems to be left in a muddled state of affairs for any number of reasons, 
including not only its ambiguous mission and goals, but its funding and structure as well. 
Clarifying the programs' missions, goals and objectives is the necessary first step . 

Appropriate Actors/Level of Use, One of the larger issues arising from this 
discussion concerns the level of use or the appropriate entity for use of the standards. 
Traditionally, responsibility for human resource development has rested with the states, 
and in the 1980s, increasingly the responsibility, authority and funding/financing for most 
related human resource efforts has been lodged at that level. But, there are real barriers to 
vesting performance management functions with governors. First, there is the unavoidable 
conflict between distributional and results-oriented objectives. As elected officials, 
governors are more concerned with distribution than performance. For the governor as 
elected official, performance and distribution are one and the same. Good performance 
tends to be defined in terms of serving the right groups or funding the right providers. 
Even the bodies established to advise the governors on resource use and performance 
management, including state councils under JTPA, ES, WIN and Voc Ed, are appointed by 
and serve at the pleasure of the elected officials. 

iv 
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Second, it is unclear why governors should take the process of managing substate 
performance management seriously. Making full use of the tools provided in JTPA -- 
performance incentives and sanctions applied to standards weighted as they see fit with the 
advice of their state councils -- governors can make friends (awarding incentives) and 
enemies (sanctions applied or incentives denied), but have nothing of obvious substance to 
gain: the state's JTPA funding is unrelated to the governor's managerial vigor. Where 
governors have nothing to gain from carrying out distasteful management decisions, they 
are unlikely to do so with anything more than minimal compliance behavior. Moreover, if 
states could benefit from management decisions, there is still a real question as to whether 
doing so would be worth the cost politically. An apolitical, quasi-autonomous state entity 
may be needed at least to oversee performance management of these programs. It is 
difficult to envision governors doing so, despite encouraging signs with the welfare-related 
initiatives. Providing the governors financial incentives may be part of the answer. 

Appropriate Measures. Generally speaking, identifying appropriate measures 
of performance becomes a relatively minor technical issue once decisions have been made 
about program mission, goals, objectives and target groups. There appears to be littie 
disagreement concerning the role standards should play in program management, guiding 
policymakers and managers to more efficient and effective program operations. It is 
expected that this requires process and outcome, efficiency and effectiveness standards. 
There is an important issue concerning the appropriate combinations of measures. 

Programs have either open or restricted eligibility. ES and Voc Ed have open or 
relatively open eligibility for service, while the other programs all have some restrictions on 
who^ can be served. Even within the more open systems, there a^e groups targeted for 
service priority or participation in special programs. Programs with open eligibility do not 
have the same needs for such process-oriented measures as participation rate standards. 
Instead, they require standards which reinforce cost-effective service delivery to all comers. 
This entails combinations of standards heavily oriented towards outcomes with both 
effectiveness and efficiency measures. 

JTPA and welfare-related efforts with tight targeting require combinations heavier 
on target group measures, including participation (e.g., WIN and Food Stamps) as well as 
outcome standards. The exact mix of standards varies with the particular program context. 
With the potential overlap between the systems and their target populations, if these 
measures are designed deliberately, the use of consistent standards and incentives policies 
has the potential for rationalizing service delivery across programs improving overall 
program effectiveness and reducing unnecessary service duplication. 

Target Groups. Standards need to be differentiated by target group. JTPA Title 
IIA standards are differentiated by age, into adult and youth standards, based on the fact 
that programs for these groups have different objectives and components. The extent to 
which standards need to be tailored to in- and out-of-school youth has been debated as 
well, as have the options for giving more direction to states on use of the "hard-to-serve" 
provisions which encourage program targeting via incentives. Voc Ed has shifted much 
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more to a categorized system with special populations targeted for service with the 
federally-funded portions. It remains to be seen how effective this taii-wagging-the-dog 
approach will be without conforming provisions in the accompanying state-funded 
programs. An effective case has also been made for implementing measures differentiated 
by target group in welfare programs based on recipients' prior work history and AFDC 
experience. 

The case made here is two-fold: first, in programs geared to promoting access or 
equity of service for certain target populations, participation (and corresponding outcome) 
standards are appropriate; and, second, available findings from the evaluation research 
suggest that program effectiveness will be better promoted if outcomes and standards tied 
to them are explicitly differentiated by target group as well. 

Minor Issues 

Role of Standards, Standards should translate program goals and objectives 
into day-to-day reality and provide program operators the measures necessary for 
determining whether results are being accomplished on a timely basis. This applies 
whether the standards are tied to process or outcomes. JTPA has gone a long way to 
defining the role of standards in Title IIA. The remaining systems, including ES, need to 
come to grips with this issue in the near future if the potential for coordinated service 
delivery among programs serving similar target populations is to be realized. From the 
recent debates on welfare reform and earlier attempts to enact Voc Ed standards, there is 
reason to think this is beginning to happen. 

Adjustment Processes, Virtually all of the programs which have addressed the 
establishment of performance standards directly have provided for adjustment processes. 
In recent years, JTPA has tackled this one issue energetically, making use of considerable 
technical resources. ES has taken similar steps but less actively; Food Stamp programs 
have explicitly incorporated modifications to required participation standards. The other 
programs have only considered tlie prospects informally. The issue which merits attention 
here has three parts: first, should adjustments be made; second, to what degree should 
adjp^^tments be made; and, finally, what is the preferred mechanism for adjustment? It is 
not wiear that the use of adjustments for any of these programs needs to be carried quite as 
far as it has been to date. Both the JTPA and ES systems - the ones in use the longest - 
have grown so complex and been modified so often that the adjustment processes have 
remained beyond the grasp of most and nearly unintelligible to the average person. This 
adversely affects the credibility of the standards. Moreover, to the extent that the private 
sector is involved in oversight and responsible for performance via the PICs, there seems 
to be son;e difficulty relating to adjustments for so many different factors md conditions. 
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Of all the programs, JTPA has far and away the most coherently designed and 
implemented performance management system. It also seems to be in need of some mid- 
course corrections, which might include: giving the'states" governors some real incentive 
for taking performance management seriously, like incentive awards at the state level; 
reducing substantially the number of standards in use; maintaining the same standards in 
place for a 2-year planning cycle; developing and adhering to a far simpler adjustment 
model implementing mechanisms to more effectively promote service to the hard-to-serve; 
and others. Note that, all of these coixections/suggestions aside, the system of job training 
has undeniably shifted from a process- to a performance-driven mode in a matter of a few 
short years. Refinements are in order; not wholesale restructuring. JTPA's performance 
management system serves as the "standard" against which the others are assessed for 
better or worse. It is the only one which has been designed, implemented and tested. 

Program-Specific Recommendations 

JTPA Title III Dislocated Workers Program . There appears to be no reason to 
proceed with what is largely a state/local charade regarding legislatively-mandated 
perfomjance standards for Title III. There is no compelling reason to have them or to use 
them. Displaced worker programs are inherently ad hoc, temporary efforts to assist in 
situations including plant shutdowns and mass layoffs. The goals and objectives of the 
program are so poorly articulated that it would be difficult to design them. And, no data 
base cuixently exists to devise them. It would appear preferable to substitute legislative 
provisions requiring or encouraging funding only providers of "demonstrated 
effectiveness." 

Postsecondarv Voc Ed prn^am<;. Establishing Voc Ed ^-^andards compaable with 
the other human resource efforts also looks nearly intractable under the current 
configuration. Counting all federal, state and local dollars involved, the public as a whole 
has far more at stake in Voc Ed than in the other self-sufficiency programs. There is very 
little accountability incorporated into Voc Ed at any level, and given the dominance of state 
and local funding, it is difficult to see federally-mandated changes making the difference. 

General Recommendations 

From ih^ federal vantage point, there are several important recommendations, 
including: 

♦ Quasi-auionomous state councils should be created with broad 
performance oversight responsibility for all human resource programs. In 
order to de-politicize the process of managing program performance, quasi-autonomous 
councils need to be created by federal legislation with broad oversight responsibility 
reaching across these programs. Membership on the councils would need to be broadened 
to provide appropriate representation for all major groups affected. Although detailed 
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options for implemeniaiion should be left to the states several features are criticaL Councils 
should: 

Have members appointed by governors, with terms which overlap .\nd 

extend beyond that of the governor, so that no governor would possess 

complete control over the councils* makeup and decisions; 

Have a staff independent of the governor or any of the prof ^ams or 

agencies; 

Have oversight and evaluation responsibilities only; they should not operate 
programs or be involved in operational management decisionmaking; and 
Hold open meetings and issue regular reports on program performance to 
the governor, the state legislature, the agencies and the responsible federal 
agencies. 

The key functions of these councils would be to: 

Review program performance for all human resource programs operating 
with federal funds in the state; 

Advise the governor and the agencies on coordination and on establishing 
state-level performance standards for these programs, including processes 
appropriate for adjusting these standards at the sub-state level; 
Recommend incentives and sanctions policies for the.se programs to the 
governor and the agencies; and 

Prepare and issue regular reports on program performance, both within and 
across programs. 

• Incentives should be instituted to reward governors for good 
program performance at the ^tate level It is important that, if governors are going 
to continue to shoulder performance management responsibilities for human resource 
programs, they have tite incentives for djing so. Just as sub-state entities under JTPA and 
ES receive the bulk of their annual funding based on need and a small but meaningful share 
(from 6-10 percent, roughly) based on performance, governors should operate under a 
similar rewards structure. Those governors who take perfomiance management seriously 
shou''^ reap ihe rewards for their states; those who do not, should not. 

• There needs to be a general tightening of what is measured as 
program performance. In most of these programs, there is very weak definition of the 
outcomes desired and almost no quality control to ensure that reporting is accurate and 
consistent within, much less between, programs and states. Further, the JTPA practice of 
defining the national departure points for standards at a level so that 75 percent are expected 
to exceed them is very discouraging. If the desired return for expending federal monies is 
employment with earnings sufficient to support families above the poverty line, then does it 
make good policy sense to allow states to define placements (or enrollments) more loosely? 
States and local entities can always opt to define outcomes more tightly than the federal 
definitions prescribe, particularly via the performance contracting process. 
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The corollary to this recommendation is that the tool kit available to the governors 
and their agencies may need to be augmented. It is decidedly contrary to market-based 
principles of operation to tighten up performance expectations while handicapping the 
responsible panies. Governors and local programs have been given responsibility for 
meetitig standards and local programs (in JTPA and ES) have been offered fmancial 
incentives for doing so, but they have also been handed a restricted set of program 
treatments and constrained in terms of the amounts they can expend for administration and 
support services. 

Programs should move toward utilization of both outcomes- and 
process-based performance standards. While recognizing that tb'ire are important 
differences between these programs, all at least share the goal of moving eligible 
participating families or their individual members into unsubsidized employment. Such 
outcome-based standards provide management the day-to-day tools for assessing 
performance, particularly where the measures implemented are proxies of the longer-term 
net impacts desired. Thus, combining the two types of measures - at least for those 
programs with restricted eligibility or important targeting requirements ~ is desirable. 

Specific state-level recommendations are as follows: 

• Governors should take the lead in giving clear, unambiguous 
direction to these programs in their states and in defining the role which 
standards should play in getting them there. An issue which has surfaced for 
several of these efforts is the ambiguity of mission and goals. In the absence of national 
guidance on this count, governors need to take the lead in clarifying the direction the 
programs are to take. Without clear missions and goals, it is not realistic to place 
substantial reliance on standards and performance manf'gement in getting there. Governors 
can and should play this role to the extent they can under their state structures. 

• States should take a comprehensive approach to rationalizing human 
resource service delivery, making use of performance standards as a 
primary vehicle. States, especially the governors, need to take a more comprehensive 
approach to the delivery of human resource development programs, rationalizing delivery 
of services across programs and among provider agencies, based solidly on performance 
against standards. Governors already have the leeway to carry out much of what is 
needed, but with the added support provided by the state councils recommended above, far 
more could be done. The first steps might include initiating and managing the process of 
standardizing definitions for key program terms, eliminating existing disincentives in state 
policies, etc. 

Finally, there are a few general recommendations, as follows: 

• Efforts to design and implement performance management systems 
need to be fully cognizant of the programs' grant-in-aid nature. The sharing 
of governance among federal, state and local actors is dissimilar across program lines, 
much of which can be attributed to varying shares of federal funding. The same 
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expectations for performance management do not and cannot apply to ES with 100 percent 
federal funding and state-controlled local offices, and to postsecondary Voc Ed with less 
than 10 percent federal funding and autonomous local entities. Expectations must conform 
to reality, not the other way around. 

Simpler adjustment models for performance standards need to be 
implemented, consistent with the preference for market-oriented 
approaches. It has become common in tl^ existing programs to build in adjustment 
processes for performance standards which take into account any number of factors, such 
as local economic conditions, the mix of people served and program activities provided, 
etc. The proposed welfare reform bills outline similar processes. Yet, these types of 
adjustments are in conflict with more typical market-based approaches. V/hile this market 
bias may not be completely appropriate for human resource development programs which 
deal with the market's failures, it is unlikely to yield any time in the immediate future. 

Models need to be developed which allow and even encourage the type of head-to- 
head competition which prevails in the private sector and which surfaces from time to time 
in these programs as well. Such competition seems to be inherent in the systems and might 
not be detrimental as long as outcomes are adjusted to some degree for conditions outside 
management's control. Features of these simplified processes include: 

Adjustments need to be few and intuitively sensible (valid); they should be 
made where there is a clear, empirical relationship to the outcome; 
Adjustments should be easy to understand and explain; and 
Adjustment models should also remain relatively stable, changing little over 
time to impart credibility and a stable operating environment to the extent 
possible for the programs. 



I. INTRODUCTION 



A. Performance Management in Human Resource ^development 

In recent decades, there has been a marked rise in the concern with performance 
management in both the public and the private sector. On the private side, there has been a 
growing uneasiness over the extent to which privf^te firms, now multinationals and 
transnationaib locked into intense international competition, are performing well enough to 
maintain our competitive position in the world. A large part of this concern has ultimately 
come to focus on the quality and capability of U.S. business management, l These same 
worries have extended to the public sector, particularly to those publicly-funded programs 
charged with the development of human resources. 

The emphasis on performance management to promote public sector accountability 
has taken various forms and directions. In some instances. Congress has mandated that 
programs perform periodic program evaluations, while in others, it has gone as far as to tie 
funding in one year to performance in the prior year or even to prescribe the detailed 
management responsibilities of the actors in the system. The evolution of performance 
management in the public sector has been neither steady or consistent: key management 
provisions have often been tightened in one program even as they were being loosened for 
another. However, more recently the need to stress program management to produce the 
desired outcomes has been felt more consistently across the board. No doubt this 
phenomenon can be explained by the twin pressures of a more informed, demanding 
public, coupled with ever-tightening budgets at all levels, especially those closest to the 
taxpaying public. The 1980s version of federalism has pushed program management 
decisions closer to home as the funding to provide services has declined. Few program 
initiatives surface these days without the obligatory accountability provisions, generally 
with increasing prescription about who should do what .to whom on which time line. 

This paper is specifically concerned with design and implementation issues 
surrounding performance standards and incentive systems in the nation's human resource 
development (HRD) programs. The programs which receive attention here ~ all of which 
are federal grant-in-aid programs with differing shares of federal funding ~ are: job 
training, job placement, postsecondary vocational education, and a number of welfare and 
related programs. Bri^f consideration is also given to the leading proposals for reforming 
the federal/state welfare and welf^e-to-work system, since performance management and 
standards are emerging as major issues in that arena as well. 

The notable exclusion from this group is non- vocational education. Education at all 
levels, both public and private, clearly fonns the backbone of HRD in th^s country. As a 
result of the excellence-in-education movement, most states have undergone education 



^For example, see: Peters and Waterman (1982). 
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'reform to some degree. Public demands for accountability in education has heightened 
awareness of educators ana pushed that system towards performance standards of a son at 
the federal, state and local levels. That process is all encompassing, ranging from teacher 
certification and curriculum approval to student achievement testing, topics well beyond the 
scope of this paper.2 In addition, the mission of the educational system is much broader 
than that of the others explored below. Education at its most basic level aims to prepare 
individuals for effective citizenship 'i a democratic society, as well as to provide paths to 
career development. 

The programs described here include among their major goals job placement in the 
relatively near-term and self-sufficiency over the longer-term, distinguishing these 
programs from education more generally. It is also the case that public education is 
ostensibly designed for everyone, while most of the programs here are not. Although there 
might be some question about the inclusion of postsecondary vocational education, those 
institutions undoubtedly serve as key service providers in the employment and training (and 
the broader HRD) system across the country. Programs considered in this paper are 
means-tested or targeted on groups with special needs, such as those seeking employment. 

B . The Management Context: Public-Private Sector Differences 

It is important to acknowledge some of the major differences between the 
management context in the public as distinct from the private sector. These differences are 
all-important but too often minimized or overlooked altogether in the rush to apply private 
sector - 'market-based', or 'bottom-line' - management principles to public institutions 
and systems viewed as poorly run.3 There has been a rush to embrace private sector 
approaches to managing publicly-funded programs, extending beyond political party 
affiliation and across ideological lines. What has resulted may well be an over-emphasis on 
the private jectcr approach. The degree to which applying these principles is appcopriate in 
the public sector ^vhere goods or services are being provided because there has been a 
market failure, that is, for one reason or another, the market has underallocated resources to 
this area or that. This is not to suggest that better management of the programs is not 
needed or desired, but that the means for reaching that end may not be exclusively market- 
based. The tendency for U.S. business to be fixated on short-term market profits has 
received severe criticism as leading to shon-sighted decisionmaking by domestic business 
when compared to our major European and Japanese competitors, the same competitors 
who have ied the way in human capital investment as a means of improving productivity. 
There needs to be some balance in the approach taken, and it should seek to fit the context 
within which it must operate. 



( 

^For just one example, see: Lewis (1986). 

3For examples of the growing literature on the application of private management principles to Fublicly- 
fundcd programs, see: Bailey (1988); Behn (1987); Elmore (1979); -he classic piece by Schultze (1977); 
and Williams (1980). 
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Major differences in the public grant-in-aid context include the following:"^ 



PQ wgr and responsibilities are shared across jurisdiction^ in the public 
sector. The degree to which these are shared by the federal, state and local 
entities is related |o the amount of funding contributed by each. 

Rather than having a central concern with performance in terms of results or 
outcomes, the public sector's focus traditionally has been on compliance 
and distributional asnenr^. Public sector officials at all levels are far more 
attentive to organizational health and comfort than to longer-term program 
outcomes. After all, the responsible parties may or may not be around to 
capitalize on the returns from improved performance, despite the fact that 
benefits would accrue to the taxpaying public. 

In the decentralized public sector, particularly with greater reliance upon 
block grants, it is more a question of influence than control . There are a 
number of key implementors possessing wide discretion in operating 
programs, many of whom are well beyond the actual control of the federal 
agencies. While the rhetoric is control, the reality is influence: the federal 
entities ~ and the states vis a vis the locals - bargain for results. Moreover, 
without a serious commitment at the level of government immediately^ 
below, there is really no way to compel policy or program change.5 

Almost by definition, the public sector lacks the pressure points found in the 
private sector, leading to a search for market-like controls . If there were an 
obvious 'bottom line*, clear delineation of roles and responsibilities, and 
near-perfect information, services would be provided almost entirely by the 
private sector. 

There are inevitably many masters often with conflicting goals . Unlike the 
private sector where there is usually one, overarching goal for the 
organization shared by all managing actors, in a grant-in-aid effort, one 
level of th^ system may be more concerned with results while another - 
typically the ones closer to actual delivery of services - is concerned largely 
with the distribution of services to key groups and areas. 

When the private sector is actually brought into the managing partnership sharing 
responsibilities in a grant-in-aid program as in the Job Training Partnership Act (JTPA) and 
the Employment Service (ES), not only does the number of actors increase, but the 
partnership becomes that much more uneasy: there are conflicting management 
expectations and approaches, as well as conflicting goals. 



"♦This discussion rclitb heavily upon trcaimenls contained in Reagan (1972); and Williams (1980). 
^Ingram (1977). p.521. 
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C. Key Cross-Cutting Issues 



Six cross-cutting issues form the basis for the discussion of HRD program 
management concerns which follows. These are issues which must be addressed 
regardless of the particular program or the level of the system, that is, federal, state or 
local. They need to be resolved whether the program is longer-term job training for the 
unemployed or short-term placement services for the general public. 

The key cross-cutting issues are as follows: 

• Legislative Purpose; 

• Level of Use; 

• The Role of Performance Standards; 

• Appropriate Measures of Performance; 

• Target Groups; and 

• General Implementation Issues. 

D. The Paper's Focus 

The paper examines program performance standards, incentives and sanctions in 
HRD prograniS from several perspectives. Cross-cutting issues are emphasized because of 
the perceived commonality of their missions: u.I are grant-in-aid programs funded 
primarily by federal and state governments for the development or more complete utilization 
of human resources. Ultimately, all have an employment outcome focus. A better 
understanding of the manner in which performance management is handled across these 
related programs should enable federal and state decisionmakers to rationalize the delivery 
systems, especially in times of scarce resources.^ The issues discussed are largely 
conceptual or design issues. Implementation issues (e.g., data collection requirements, 
etc.) are brought into the discussion but are not the primary focus. The research has relied 
both upon surveys of the existing literature, as well as in-depth interviews with other 
researchers, national and state policymakers and program administrators. 

Section II briefly describes each of the programs, the evolution of their particular 
performance management requirements and the nature of their current management 
systems. Section III develops the cross-cutting issues more fully, while Section IV 
analyzes the programs in terms of these issues. Section V provides a synthesis of the 
issues, major and minor. Section VI departs from the lessons learned from the extensive 
experience with standards and incentives with JTPA and ES, making recommendations for 
federal/state policymakers and program administrators. 



^For a brief but provotalivc draft paper examining the principles undcrl>ing the possible rationalization of 
the broader employment and training system with an eye towards performance concerns, sec: Wallace 
(1983). 
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IL THE PROGRAMS AND THEIR PERFORMANCE MANAGEMENT 
SYSTEMS 



This section describes each of the programs, including a relatively detailed account 
of its current system for managing perfonnance and a short history of its evolution. The 
programs reviewed include: job training as embodied by the federal/state/local system 
established under the Job Training Partnership Act of 1982; job placement or the ES, 
authorized by the Wagner-Peyser Act of 1933; postsecondarv vocational education or Voc 
Ed, lecently reauthorized under the Carl D, Perkins Vocational Education Act of 1984; and 
a number of welfare-to-work programs , ranging from the Work Incentive (WIN) Program 
for recipients of federal/state Aid to Families with Dependent Children (AFDC) to the Food 
Stamp Employment and Training Program, Figure 1 provides summary information 
concerning these programs. Brief attention is also given to current proposals for welfare 
reform, each of which contains standards provisions. 

A. Job Training 

The publicly-funded system for job training is authorized currently by the Job 
Training Partnership Act of 1982 (P.L. 97-300, as amended). JTPA is designed to 
provide training and, to a much lesser extent, employment largely for economically 
disadvantaged adults and youth. With almost $4 billion in federal appropriations in Fiscal 
Year 1987, a mix of activities and services is available under three major operational titles: 

• Title II: Training Services for the Disadvantaged : Under Pan A, activities 
including classroom training (occupationally-specific and basic skills), on-the-job 
training, job search assistance and limited work experience are offered by some 
600-plus local Private Industry Councils (PICs) to economically disadvantaged 
adults and youth. Under Part B, PICs operate summer employment programs for 
eligible youth, aged 14-21 years. With the 1986 Amendments, summer programs 
increasingly emphasize remediation-oriented activities (reading anJ math skills 
acquisition), as well as more traditional part-time employment. In FY 1987, $2.6 
billion was appropriated for Titles IIA and IIB. 

• Title III: Employment and Training Assif;tance for Dislocated Workers : Workers 
displaced from their occupations and industries through plant closings, mass 
layoffs or other actions (e.g., disaster, federal actions) were provided many of the 
same training and related services as under Title IIA with FY 1987 appropriations 
of $200 million.'^ The administrative and programmatic variations under Title III 
are more varied than those under Title II, in part because the funds flow directly to 



^Thc alicmpi to replace Title III with a rcsiruclurcd Worker Readjustment AssisUnce Program at greatly 
enhanced funding levels as pan of the Omnibus Trade Act met with a Presidential veto which was 
overridden by the House but sustained by the Senate in early June 1988. 



Figure 1 
Programs Overview 



Primary 
Mission 



JIEA 

• Increasing employment 
and earnings of 
disadvantaged youth, 
adults and dislocated 
workers through job 
training. 



ELS 

• Labor exchange 



VO C E D* 

Unclean market 
and non-marka 
oriented mission 
and goals. 



ma 

• Reducing dependency 
and promoting self- 
sufficiency for AFDC 
recipients-through job 
trainin;- and employ- 
ment. 



FOOD STAMPS * ♦ 

• Reducing dependency of 
Food Stamp recipients 
through short-term 
interventions. 



Target 
Groups 



Title IIA 

Economically disadvan- 
taged adults and youth 
Welfare recipients 
School dropouts 

Title III 

Dislocated Workers 



Open eligibility 
Priority to: 

- Veterans 

- UI Claimants; 

- Handicapped; 
-"Groups with 

special needs." 



Disadvantaged 
HandicaRXJd 
Single parents/ 
displaced home- 
makers 
Offenders 



Voluntary AFDC 
caretakers 

Mandatory registrants 
not exempt due to: 

- children under 6; 

- ill, incapacitated; 

- age (under 16, over 65); 

- other. 



Voluntary regisuants 
on Food Stamps. 
Mandatory registraints 
not exempt due to: 

- WIN regisuation; 
-age; 

- ill, disability; 

- other. 



Activities 
& Services 



Classroom Training 
OJT 

Work Experience 
Job Search Assistance 
Support Services 



Counseling and 
assessment 
Labor Market 
Information 
Job Referrals 
Job Development 



Occupationally- 
Specific Education 
Cooperative education 
Work Study 
Apprenticeship 
Support Services 



• Training 

• Work Experience 

• Job Search Assistance 

• Grant Diversion/ 
Work Supplementation 

• Support Services 



• Primarily short-term 
Job Search Assistance 
or Training. 

• Support Services 



FY 1987 $2.6 billion (Title IIA & B) $778 million $220-306 million (est.) $1 10 million $50 million 

Federal $220 million (Title III) 

Funding 
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governors with only minimal guidance as to how local efforts are to be structured. 
PICs, coniinunity colleges, labor organizations and private-for-profit firms have all 
played a mrjor role in seivice provision at the local ievel, as have state agencies. 

♦ Title IV : Federally Administ ered Programs : Three federally-administered 
programs are supported render Title IV, including programs for Native Americans 
and Migrant/Seasonal Farmworkers under Pait A, the Job Corps under Pan B, and 
Veterans under Part C. In FY 1987, these effons received a total appropriation of 
$750 million. In some instances, these programs are operated through local PICs; 
more often, they are not. 

Performance standards including incentives and sanctions now play a prominent 
role in training programs for adults and youth operated by PICs under Title IIA.' Their 
intended -ole is more ambiguous in Title III and much more so under Title IV. Given the 
dominant role of the Title II and III programs in the federal scheme, the discussion here 
vAll center on those efforts to the exclusion of Title IV. It would be a serious slight to the 
Job Corps programs to pass over them completely, aowever: the most persistent efforts to 
build perfo-mance management into any HRD effort in this country are probably found in 
that system, from top to bottom^. In Job Corps, performance against standards has been 
factored directly into the procurement process for selecting program operators. Job Corps 
is a smaller, more tightly controlled system, with a du-ect contractual relationship from the 
Federal level, targeted upon a narrower clientele. Joj Corps will net be incorporated into 
this discussion. 

Job Trainin g Performance Management: A 'Rnef History There have been three 
distinct phases in the development of performance standards and accompanying 
management components for job training progra.iis.iO In the fir^t phase, dating roughly 
from the emergence of U.S. employment and ti-^i i'ng policies ir .961 to the passage of 
comprehensive manpower legislation in 1973, programs for job training were inherently 
performance-based, at least on one level: only programs whose staff wrote quality 
proposals and demonstrated a record of past performance were funded to provide service a. 
With few exceptions, very little attention was given to program management at the state or 
local level or to program results beyond the point of the initial job placement. Funding was 
based largely on local 'grantsmanship' and need for services. B> the late 1960s, in 
keeping with the Great Society's thrust, policies and programs stressed services to 
numerous target populations far more than participants postprogram success. 

The second phase began with the passage of the Comprehensive Employment and 
Training Act (CETA) of 1973 and ended in 1983. CETA brought significant changes to the 
job training system, not least among them, being that funding became tied directly to 
measures of need. Areas with greater relative shares of the economically disadvantaged 



^For discussions of JTPA performance standards and incentives, see: National Governors' Association 
(1983. 1985); Ryan (1986); King and Pitlman (1987); and Strumpf (1985). among others. 
^For exantple, see: Johnson (1985). 

l^For a Lengthier treatment, see: King and Pittman (1987). pp. 12ff. 
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and the unemployed received more funding to provide services, regardless of their record 
of performance. CETA was among the first of the revenue sharing initiatives ot the early 
1970s. Only extreme fiscal or legal problems could keep an area from receiving their 
annual funding: funding was actually denied to only one area (East St. Louis) during this 
period. 

The Department began an internal effort in the mid-1970s - with participation from 
federal, state and local staff - stressing management of performance-orienied programs. 
Goals and expectations were established, including groups of measures ("performance 
indicator clusters") by which federal staff in the regions could assess local programs based 
on performance. Federal staff were to make adjustments in the indicators based upon a 
series of interpretive factors, taking into account the degree of difficulty of groups served 
and the severity of local economic conditions. The process was subjective, varying by the 
individual federal staff pe;.>on. No significant implications were tied to performance. It 
was an interesting beginning, but, without true rewards and sanctions, the process suffered 
a lack of credibility. 

The passage of the 1978 CETA Amendments significantly increased the attention to 
performance management in a number of ways, including the addition of a far more explicit 
statement of purpose mandating that the Secretary establish performance standards (taking 
into account the characteristics of those served, economic and geographic differences, etc.) 
and creating a national Office of Management Assistance. Most of the features of the 
current system were initiated in response to these amendments. The legislation stopped 
short of mandating a complete management system, and a substantial distance remained 
between legislation in Washington and practice in the field. While standards were 
mandated, it took several years before the methods of analysis and the program data to 
support ihem could be developed. Trial' standards were introduced in 1982, but before 
full-blown implementation, there was a complete legislative overhaul. Moreover, the 
legislatively-mandated management assistance capability was never funded. Despite the 
changes, there were no clear implications for local performance. 

The third and current phase began in earnest with JTPA's implementation in 1983. 
JTPA instituted a number of important performance-related changes, including: 

• Explicit recognition that training was "an investment in human capital;" 

• Vesting of clear and unambiguous responsibilities for oversight and 
management at the state level with the governor and with a partnership of 
the PICs and local elected officials (LEOs) at the local level; 

• Shifting the program's focus from process to outcomes, mandating that the 
measures of perfonnance for the Title IIA program for adults be increased in 
employment and earnings and reductions in welfare dependency and that 
both placement and non-placement measures be utilized for youth; and 
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• Providing for financial incentives and sanctions for failure to perform 
against standards; real significance was attached to local program 
performance. 

JTPA is funded locally based on a combination of need and performance, and the 
^tv ,'nior has explicit responsibility for ensuring that local service delivery areas (SDAs) 
perfomi under the guidance of their PICs. The system outlined in the Act has the potential 
for offering a well-defined, balanced system of perfomiance management at all levels. 
There are any number of possible approaches consistent with the legislation. >1ost 
observors of the current system agree that it is performance-driven. The question may be 
whether it is too performance-driven, pushing program operators to trade off service to 
groups most-in-need in favor of those easiest-to-place. 

JTPA Performance Management . The performance management system 
surrounding the provision of programs for adults and youth under title IIA has been 
developed incrementally since 1983 with intensive input from state/local entities and public 
interest groups. It is a hierarchical system, with roles and responsibilities for federal, state 
and local actors differentiated in rough relation to their distance from service provision. In 
keeping with legislative intent, the system is largely performance-driven . emphasizing 
outcomes - both at-termination and post-program - over in-program measures. There are 
both positive rewards and negative sanctions. 

The essential roles in Title IIA are as follows. The Secretary sets the basic direction 
for the program nationally, including the numerical performance standards, and prescribes 
the rules of the game for governors and others, e.g., policies for varying perfomiance 
standards, implementing incentives and sanctions, performance contracting, etc. 
Governors, with advice from their State Councils, set the "real" standards for programs in 
their states. From a policy and program management perspective, governors are pivotal in 
the standards implementation process. Among other critical decisions, governors may: 

• Add standards in addition to the Secretary's standards; 

• Determine the relative importance accorded each standard; 

• Vary the numerical standards facing the local PICs; 

• Shape the contracting process within the state; 

• Define and establish specific incentives for service to the hard-to-serve; and 

• Develop performance incentives and sanctions policies. 

These are the decisions which give life to the policies outlined in the legislation and 
federal regulations. PICs make the final translation into 'bottom-line' performance criteria 
locally for service providers. The device for this is typically two-part: the funding 
application package and the contract, most of which are now perfomiance-based. It is the 



cbniracis which are the systems' true drivers. Rhetoric aside, if placement targets are the 
sole measures included in PICs' contracts with providers, then placement is the only real 
standard. Increasingly, PICs are sharing their rewards with service providers, passing 
along some portion of the incentive awards to those most closely associated with the 
performance which secured them. 

The guiding principles of the performance standards framework under JTPA Title 
IIA can be stated relatively simply. These principles have evolved from several years of 
discussions at all levels of the system in developing the standards and incentives/sanctions 
policies, dating from the latter days of CETA. The basic approach stemmed in part from 
observation of private sector management practices in firms with decentralized 
responsibility and profit centers. They embody much of the management philosophy 
behind JTPA performance management. The guiding piinciples, many of which apply to 
ES as well, are: I' 

• Performance standards should be used to encourage good performance, SDAs 
should not be penalized for serving groups which are particularly difficult to serve, 
such as welfare recipients, dropouts and others. 

• SDAs should be rewarded for good performance, not good planning. 

• SDAs should net be penalized for operating in a difficult environment, 
panicularly one undergoing major economic and labor market distress. 

• An SDA should primarily compete against itself, not other SDAs. 

^ • Performance standards should serve as a management tool for use in program 
planning and evaluation. 

• Numerical standards should serve as the beginning point for state-local 
discussions concerning performance expectations, not the ending point. 

• There is much more to good program performance than just exceeding 
performance standards. 

Under Title TIT, the management system has been given little shape or form, in large 
part because the roles of the actors and the nature of the programs to be established have 
been left unclear. The Act creates a system within which the Secretary at the federal level is 
required to set standards based on placement and retention for programs which range from 
short-term, outplacement services in a single-plant shutdown in a rural area to traditional 
classroom training efforts for large numbers of unemployed workers. At the state level, it 
is important to note that the governor may or may not pass through Title III funds to local 
PICs. Some states retain significant funding at the state level to support Rapid Response 
Teams for responding to emergency situations (e.g., mass layoffs or plant shutdowns^ and 



'Scc: King and Pittnian (1987). as well as the various DOL Performance Standards TAGs. 
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allocate the balance to some or all PICs by formula. Incentives and sanctions are not 
required nor is it clear what rationale exists for them. In many cases, local PICs will be 
operating one-time programs for a unique set of workers in an occupation or industry. 
Future rewards or sanctions would probably have little effect on current program 
performance. 

Performance management for Title TIB Summer Youth Employment and Training 
Programs recently underwent considerable discussion and modest change with the passage 
of the 1986 Amendments. While initially all of the actors - from the governor to the PICs 
and LEOs - had the "same authority, duties and responsibilities" under Title IIB as ^Vey 
had under Title IIA [Section 254], the authorizing legislation now contains an explicit 
statement of purpose [Sect? Jii 251] and requires local SDAs to develop written goals and 
objectives for their summer programs. This shift from the twenty-year-old focus on short- 
term employment arose from considerable research and Congressional concern thai brief 
periods of summer employment for disadvantaged youth without remediation would leave 
them as adults in pover^ later. Note that there is actually no real management system or 
even a set of management actions attached to these new requirements. The sharing of 
responsibilities is unlike that for Title HA. Governors do not have a real management role. 
SDAs retain the basic responsibility for operating summer programs - that is, recruiting 
kids, developing jobs, etc. — and current funding remains tied to need rather than last 
year's performance measured by post-progran* placement, reenrollment in scnocl or change 
in math/reading test scores. Management concerns revolve around compliance and 
expenditure levels for the most part. 

B . Job Placement/Labor Exchange 

Since ine pa^' ^ ' '''^ Wagner-Peyser Act in 1933, State Employment Service 
Agencies (SESA' v., . ^jrt offering a wide array of services - providing labor market 
information, devc.v^ *g job orders with employers, testing and screening potential 
employees, providing job referrals and performing the "work test" for unemployir ^,nt 
insurance claimants, welfare and food stamp recipients, and others.^2 SESAs have 
operated as a federal/state partnership, serving both employers and potential employees, 
facilitating the labor exchange. Eligibility for these services, unlike many of the others 
discussed in this paper, is open, for both employers and employees, although some 
categories of workers (e.g., migrant/seasonal farmworkers, handicapped and veteran-) are 
given special priority. 

Title V of JTPA significantly amended the Wagner-Peyser Act in a number of 
ways. States now receive most of their funding based exclusively on need, as measured by 
relative shares of unemployment and the civilian labor force. In addition, the federal 
allocation flows directly to governors rather than to SESAs, with funds divided into two 
components: 90 percent f< t the provision of basic lahor exchange services locally, through 
a :'tbscite funding allocate, n determiner' by the governor; and the remaining 10 percent 



'-^For an excellent descripiion of ih. -or exchange system and a discussion ofperfonnancc management 
issues, see: Chadwin, et. al. (1977). This paper is dated, but ilie treatment of tht issues h still relevant. 
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reserved at the governor's discretion for performance incentives, services for groups with 
special needs, and the added costs of exemplary models. Planning for labor exchange must 
also be conducted jointly with the PICs at the local level, and, as with JTPA, the 
operational cycle is the program year. Concerns with performance management and the use 
of standards and incentives have been on the increase i*^ ^he labor exchange system. 13 
1987 appropriations for labor exchange amounted to nearly $780 million. 

The Evolution of ES Performance Management . The recent history of performance 
management in the labor exchange system reveals frequent and major shifts in direction, 
much more so than in job training, i^ Unlike the evolution of performance management 
in job training, the phases are far more difficult to characterize for ES. In the 1950s and 
early 1960s, funds for labor exchange were allocated to SESAs based primarily upon 
workload factors ("workload time factor budgeting"). This system was dropped in favor 
of one based exclusively upon need as measured by the state's relative share of 
economically disadvantaged persons. This shift was in keeping with the redirection of the 
ES and other programs in the period toward serving the poor. Then, from 1975 to 1979, 
funds were allocated to SESAs based partially upon performance, first through the 
Balanced Placement Formula from 1975-1977 and later through the 24-factor Resource 
Allocation Formula from 1977-1979. Most of the performance measures included in the 
formulas were tied to short-term placements, leading to widespread criticism of the 
mechanism throughout the system. The period from 1980 to 1982 once again saw a return 
to state allocations based upon need, i.e., 98 percent based upon the prior year funding 
level. This allocation formula essentially aligned the states' funding proportions with their 
relative performance in the late 1970s. Since JTPA's passage in 1982, most federal labor 
exchange funding flows to states based on need. 

Perfonnance Management in ES . The JTPA amendments to Wagner-Peyser seem 
to have straightened out a few kinks in the labor exchange system relative to job training. 
Both systems now receive funds at the state level based upon need rather than performance, 
and responsibility for program management, including the quality of the outcomes, rests 
with the governor. The governor has the authority to foster coordination between the job 
training and job placement systems within the state, including establishing state-specitlc 
performance standards and accompanying incentive systems. Finally, at the substate level, 
PICs are responsible for jointly planning both job training and job placement funds in their 
SDAs. 

However, a significant hitch remains. The 1982 amendments stopped short of 
requiring the Secretary to establish performance standards at the federal level. Instead, the 
Secretary is allowed to do so. To date, there mainly has been encouragement on the 
Department of Labor's ES side. Nationally, the USES did develop state and substate ES 
models designed to establish and adjust performance standards to support state and local 
planning and management. But, states have had to proceed largely on their own, making 



^^The guide by King and Pittman (1987) discusses a variciy of issues related to standards and incentives in 
both JTPA and Wagner-Peyser. 

^^For a concise history, see: Fairchild (1980). 
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use of guides for the development of state and substate standards prepared in the 1983- 
1985 period. These were disseminated to the states, but their use has not been actively 
promoted as has been the case with JTPA. 

C. Postsecondary Vocational Education 

Vocational education, at best, can only loosely be described as a "system" at any 
level in this country.^^ Federal funding for vocational education programs is currently 
provided under the Carl D. Perkins Vocational Education Act of 1984. Secondary and 
postsecondary training is offered by a large array of providers, including both public and 
private institutions, high schools, technical institutes, community colleges, four-year 
secondary institutions, correctional facilities, and proprietary schools. As might be 
expected, there is wide variation in the scope, nature and quality of these programs. This 
discussion focuses on post-secondary programs. Voc Ed at the secondary level appears to 
be focusing more upon generic skills acquisition in recent years. Many of its programs are 
becoming integrated with so-called academic-track curricula.^^ 

Federal support for postsecondary Voc Ed began with the passage of the Vocational 
Education Act of 1963, although such support existed at the secondary level for nearly fifty 
years before. Major amendments in 1968 and 1976 broadened the definition of Voc Ed, 
strengthened the emphasis on planning, evaluation and accountability in general, 
emphasized improved access for certain groups and called for the elimination of sex 
stereotyping, among other changes. The Perkins Act of 1984, while a significant 
reauthorization, largely represents incremental change, a continuation of the existing 
process, stressing two primary objectives: improved program quality and access. In a 
significant departure, the Perkins Act promotes a highly categorized approach to Voc Ed 
programming with numerous setasides for key groups and programs. , 

Federal dollars account for only about 6 of every 100 dollars spent on 
postsecondary programs. The secondary/postsecondary funding split is a state-level 
decision resting with the state education board; not surprisingly, the share allotted to 
postsecondary Voc Ed varies widely from state to state. According to one recent survey, at 
least one-half of the states have allocated less than 20 percent of their federal funds for 
postsecondary Voc Ed. In FY 1987, the federal appropriation for Voc Ed was $875 
million, of which it is estimated that only about $220-300 million went to postsecondary 
programs. Nearly half of the students enrolled in postsecondary programs nationally were 
in occupationally-specific programs, including such traditional occupational groupings as 
agriculture, distributive education, health, occupational home economics, office 
occupations, technical and trade/industrial. A number of other Voc Ed programs which are 



l^For an overview of Vocational Education in the U.S., sec: Hoachlander, Choy and Lareau (1985). 

'^Probably ihe best cxanriple of the integrated approach to Voc Ed programming is provided by the 
approach taken in New York State. 

'^For an iniercsli .g overview of the provisions and the common ground between Voc Ed a*id JTPA from a 
state perspective, see: Roberts and Peuossian (1987). 
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more difficult to characterize are provided as well, including cooperative education, work- 
study and apprenticeship, much of which is at the postsecondary level. 

Management of PostSecondarv Voc Ed . It is important to acknowledge from the 
outset that the nature of the Voc Ed system is quite different from the others. 
Overwhelmingly, funding for Voc Ed is non-federal, altering the character of the program 
policy bargain considerably. In addition, funding (federal, state and local) ends up in the 
hands of provider institutions which answer to independent, elected boards, each operating 
with its own value system according to its own mission, goals and objectives. This federal 
grant-in-aid program sits at the other end of the continuum from JTPA: there are shared 
responsibilities, but the degree to which they are shared and the extent to which they are 
controlled is very different from the programs with predominantly federal funding. 

While there have been longstanding concerns with the performance of Voc Ed 
programs, both secondary and postsecondary. these have been largely in terms of 
clarifying the mission of the programs more generally and conducting appropriate 
evaluations of their results.^S It has been suggested recently that Voc Ed, much like pre- 
JTPA job training programs, has been inordinately process-oriented. The Perkins Act 
provides some opportunities to change that situation, yet it does not build in standards or 
add substantively to l he performance orientation of postsecondary Voc Ed. Some states 
(e.g., Florida) have capitalized on these opportunities, but most have not. ^9 

The Perkins Act contains several sections regarding performance. To receive 
federal funding, states must assure the feds in the State Plan that they will develop 
measures of program effectiveness ~ such as the occupations trained for, the levels of 
skills to be attained, and the basic employment competencies used - and that they will 
evaluate not less than one-fifth of their eligible recipients every year [Section 113]. There 
are no performance standards per se . and there is no explicit, federal/state/local 
management process envisioned beyond those few paragraphs.^O States are left to their 
own devices in terms of program performance management for postsecondary Voc Ed. In 
effect, the main control mechanism for these efforts lies in the accreditation arena, again a 
process-oriented effort, which operates outside of the funding decisions for the most part. 

D» Welfare Employment and Training Programs 

As welfare rolls have grown and federal and state budgets have tightened in recent 
decades, there has been renewed interest in welfare-to-work programs. Several of these 
are discussed briefly below. In the case of job training and placement programs for AFDC 



'^For example, sec: Planning Papcrf;.... (1979), especially the papers by Grubb; Bogetich and Lammers; 
Walsh; Swanson; and Thurow. 

^^Florida has established 70 percent placement rate standards for the targeted populations which ply to 
the federally-funded programs: failure on three of the standards precludes receipt of state vocational 
education funds. 

^^Il should be noted here that performance standards were considered during the Perkins Act rcaulhorizauon, 
but not enacted. 

14 



29 



recipients, there are a number of variations to describe, beginning with the oldest federally- 
funded program, the Work Incentive (WIN) program. 

1 . Welfare-to-Work Programs for AFDC Recipients 

After several small forays in the early 1960s, the federal government jumped 
decisively into the welfare-to-work arena in 1967 with amendments to the Social Security 
Act creating WIN.^^ For more than a decade, this program served as the centerpiece, 
providing federal funding for programs to assist recipients in obtaining training and 
employment and in moving off the welfare rolls. The program underwent a number of 
significant changes during that time, including: a shift from emphasizing training and 
support services to more immediate job search assistance and placement; a move to 
mandatory participation in 1972 for those without young children; and shifting the 
administrative responsibility for the programs from state welfare agencies to SESAs. Since 
the passage of the Omnibus Budget Reconciliation Act (OBRA) in 1981, there have been 
substantial changes in the program, especially reduced funding and increased state 
discretion, giving rise to considerable within- and between-state program variation. 

Nationally, 90 percent of WIN*s funding is federal. With FY 1987 funding at the 
$110 million level, WIN funding has declined by more than 65 percent in actual dollars and 
ne^ly 78 percent in constant dollars since FY 1979. WIN has never received sufficient 
funding to do much more than comply with program "registration" for most participants 
even in years with far greater funding: only half of WIN registrants actually receive 
significant services. 

States now have far more flexibility to design and operate programs for AFDC 
applicants/recipients.22 Four program options have been made available since 1981, 
options which are not mutually exclusive: 

♦ WIN Demonstration . Under this option, governors have the opportunity to apply 
to have a single state agency maintain responsibility for all aspects of the state's 
WIN program. Twenty-six stat^"^ are now operating such programs. 

♦ Community Work Experience Programs (CWEP) . States may require recipients 
to participate in CWEP as a condition of benefit receipt. The main activity under 
CWEP has been "workfare," where recipients work off the value of their grants in 
subsidized public or private-non-profit employment. Twenty-six states have opted 
to run CWEP (though not the same 26 operating WIN Demo). 



21 This description relies on an excellent discussion of past and present work-\\^iiure programs found in 
Nightengale and Burbridge (1987). For additional information on these programs, see: Smith (^987). 
220BRA created the first three options in 1981 (WIN Demo, CWEP and work supplementation. The Tax 
Equity and Fiscal Responsibility Act (TEFRA) of 1982 allowed state to operate job search programs for 
AFDC applicants as well as for AFDC recipients. The Deficit Reduction Act (DEFRA) of 1984 expanded 
the grant diversion option under work supplementation to include private sector jobs. 
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• Job Search. Since 1984, states may also require AFDC recipients and applicants 
who are required to register for WIN to participate in group or individual job search 
by offering such programs on a. statewide basis. Twenty-five states are operating 
such programs (referred to as Title IVA Job Search). 

• Grant DiversionAVork Supplementation . States may also establish grant 
diversion/work supplementation programs where the recipient's grant is diverted to 
either a public or private employer to subsidize the costs of OJT and wages paid. 
Only 15 states operate grant diversion programs. 

The Evolution of WIN Performance Management . Prior to the 1981 revisions, 
,WIN programs were managed jointly through a dual organizational structure relying on 
both Labor and Health and Human Services staff at the federal (national and regional) level. 
In general, states followed the federal pattern with separate management units in both the 
central welfare and employment agencies and in local offices as well. There was typically a 
high degree of coordination between the agency staffs in the planning and operation of the 
WIN programs, particularly those with co-located staff at the state and local levels. The 
expressed mission of the program was to find jobs for as many WIN registrants as possible 
while increasing the wages and duration of those jobs. 

In this pre-1981 penod, the key federal/state/local communication of performance 
expectations was via the .ormula for distributing program funds. WIN resources were 
distributed based on a complex formula combining need (or prior service levels) and 
performance. Performance was measured in terms of placements, wages, job retention, 
and projected welfare grant reductions, taking into account the degree of difficulty of 
serving the AFDC population. Poor performance directly affected future state funding 
allocations. Increases in both program options and state administrative discretion since 
1981 make describing the existing federal/state management structures and the performance 
management system in any meaningful way almost impossible. State discretion is high, 
but structured, regular documentation and reporting regarding the use of this discretion 
remains low. 

Despite considerable attention given to evaluations of work/welfare program 
interventions for AFDC recipients over the last several decades, translation into operational 
performance standards, incentives and sanctions has been lacking.23 More recently, 
researchers, policymakers and program administrators have begun to focus on the need for 
standards in the management of these efforts, particulariy in the on-going debate over 
welfare reform.^^ 



o 

an example of research into the factors affecting performance and the indicators of local program 
performance, see: Milchell, Chadwin and Nightengale (1980). 

arc a seiies of papers available concerning design and operational issues surrounding performance 
standards for welfare and related programs, primarily AFDC and General Assistance, in ilie context of JTPA 
standards. See: Kogan, ei. al. (1984, 1985). 
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2. Food Stamp Employment and Training Programs 



Until recently, there have been few structured employment and training programs 
for Food Stamp recipients. Work registration requirements were instituted for Food Stamp 
recipients in the early 1970s, gradually evolving into programs consisting of funded 
.activities and services for those recipients.25 However, until the Food Security Act of 
1985, there were few substantive initiatives beyond the pilot workfare demonstrations 
authorized in 1977 and again in 1981. The 1985 legislation required that all states 
implement mandatory employment and training programs for recipientb nut already subject 
to work requirements under other federally-funded efforts (e.g., WIN mandatories) by 
April 1, 1987. 

This new program for Food Stamp recipients, administered at the federal level by 
the U.S. Department of Agriculture's Food and Nutrition Service (FNS), is to be designed 
and operated by the state welfare agency. States have wide discretion in how these 
programs ai'e to be run, including the activities and services to be provided and the choice 
of service providers. It is significant that these programs, for the first time, carry 
participation standards, expressed in terms of percentages of the eligible population which 
must be served in the programs. Outcome standards have been discussed but not 
implemented. Because of limited federal funding - $50 million in FY 1987, rising to $75 
million over the next four years - little substantive intervention can realistically be 
expected: it has been estimated that the FY 1986 funding level ($40 million) allowed a per- 
recipient expenditure of about $26, barely enough to cover the registration process, much 
less the provision of significant employment and training services.^^ 

3, Welfare Reform Initiatives 

It is interesting to note that the leading proposals for welfare reform contain explicit 
language on performance management, including provisions on program mission, goals 
and objectives, as well as participation and performance standards. Only two bills have 
survived the legislative debate, one in the House and the other in the Senate: The Family 
Welfare Reform Act [H.R. 1720] passed the full House in December 1987 and is known as 
the Ford-Downey bill; and The Family Security Act [S. 1511], or the Moynihan bill, was 
passed by the Senate in June 1988. (The Conference Committee has recently begun the 
work of reconciling these bills into a single bill which can be sent to the President.) Both 
would effect significant change in the income maintenance aspects of the current system, 
focusing attention on the adequacy of state benefit levels and mandating two-parent 
programs, but many of their provisions are aimed at increased emphasis on operating 
effective programs for recipient self-sufficiency. 



^^For an overview of the evolution of employment and training programs for Food Stamp recipients, see: 
Urban Institute (1986). and Nightengale and Burbridge (1987). In addition, for a concise treatment of the 
research findings on Food Stamp and related programs, see: Auspos (1986). 
26Guei 3n and Sherwood (1986), p. 3. 

17 



H.R. 1720 would establish the National Education, Training and Work 
(NETWORK) Program, a more detailed and prescriptive effort than the counterpart 
proposed in S. 1511, the Job Opportunities and Basic Skills (JOBS) Program. 
NETWORK mandates client-agency agreements and service plans to be overseen by case 
managers, and the list of activities and services prescribed is more detailed than under 
JOBS. In addition, the system of performance standards and incentives in H.R. 1720 
[Section 436] parallels that in JTPA. Performance standards would be established wirfl 
state-level variations, targeting on priority groups, and eligibility for incentive payments 
depending upon the degree to which a program exceeded the standards. Performance 
would be measured by outcomes - including placement in jobs with adequate benefits and 
long-term job retention - rather than participation levels. On the other hand, S. 1511 
requires that performance standards be established within five years of the program's 
enactment based on program research, but does not contain further specification [Section 
204]. Both bills explicitly address standards, virtually ensuring that performance 
management and standards will be an important part of any program enacted. 
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III. THE KEY CROSS-CUTTING ISSUES 



A. Legislative Purpose 

Each program has its own mission or legislative purpose , as well as its own distinct 
set of goals and objectives. The mission is the program's reason for existing, not 
something which can ever actually be attained. The missions for these programs vary 
widely, ranging from a relatively explicit, longer-term focus on increased employment and 
earnings in JTPA to more shon-run oriented aims, such as found in the Wagner-Peyser 
Act. However, all share a mission contained under the broad human resource 
development (HRD) umbrella. The goals of HRD effons can be funher classified into 
three main areas: maintenance, development and utilization.^^ The focus here is only on 
programs concerned with development and utilization. The standards - and associated 
incentives and/or sanctions - established should be congruent with the program's goals 
and objectives. Unless the mission, goals and objectives of the program are clear, it is 
difficult to establish such standards. 

One of the attribiites distinguishing management in "excellent" corporations is that 
they possess simultaneous loose-tight propenies .^8 jhat is, performing entities are 
allowed considerable discretion in terms of how they produce, but management holds very 
tightly to what they should be striving to accomplish. Values - established and managed 
by the leadership - are tightly held, while the means for pursuing them are left relatively 
open to field decisionmaking. It is even more critical that goals, objectives and 
accompanying standards be expressed and communicated clearly in publicly-funded, grant- 
in-aid programs than in private corporations where the 'bottom line' is near term, evident 
and readily measurable. 

There is wide variation in both the time-orientation and the clarity with which the 
programs' purposes have been expressed. (Figure 2 summarizes the missions and goals of 
the programs.) Clarity is particularly imponant for program management, in that the clearer 
the goals of the program are expressed -- and understood and adopted throughout the 
system - the easier it is to lay in place a management system with workable standards, 
rewards and sanctions policies. The programs fall into three, relatively distinct groupings. 

First, two programs - the ES and Food Stamps - appear to be authorized for 
short-term resuhs and to have a limited number of unambiguous goals . These programs 
share goals more geared to near-term resource utilization than to longer-term development. 
ES, at least in terms of its labor exchange mission, was created to facilitate clearing the 
labor market, to assist potential workers (and employers) in obtaining jobs, quality 
notwithstanding [Sections 3 and 7(a)]. A small degree of confusion may have crept into 
the picture with the 1982 JTPA amendments which referred to "wages, retention and other 



^"^Harbison (1973. 1976). 
^^Pcicrs and Waterman (1982). 
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Legislative 



.TTPA 



Mission • To prepare youlh & unskilled 
adults for labor force entry 
• To afford job training to the 
cconomicaJly disadvantaged & 
others with serious barriers to 
obtain productive employment 



Labor exchange, clearing 
the labor market 



Goals & 
Objectives 



Title IIA 

Job training as "investment 
in human capital," leading to: 

ADULTS & YOUTH 

• Increased employment 

• Increased earnings 

• Reductions in welfare 
dependency 

• Increased postprogram 
employment & earnings 

YOUTH 

• Attainment of PIC employ- 
ment competencies 

• School completion 

• Enrollment in other train- 
ing/apprenticeship 

• Enlistment in Armed Forces 



Successfully matching 
jobscckers and potential 
employers, measured by: 

• Placements 

• Wages, retention & 
other factors 



3 



0 



Title III 

Placement in unsubsidized 
employment 

Retention in unsubsidized 
employment 



Q " Postsccondary. 
^|^(^* The employment and training program portion. 



Figures 2 
: Mission, Goals and Objectives 



VOC Ed ♦ 

• Unclear, ambivalent 



WIN 

• To restore families of 
individuals on AFDC to 
independence and a^ful 
roles in the community 



FOOD STAMPS ** 

• To assist members of partici- 
pating households in gaining 
skills, training or experience 
that will (promptly) increase 
their ability to obtain regular 
employment 



Improved target group 
access to programs & 
jobs 

Improved skilis 
acquisition and labor 
market responsiveness 
Miscellaneous (e.g., 
helping depressed areas, 
strengthening research, 
enhancing application of 
modem technologies) 



Unstibsidized employment 
Training for work in the 
regular economy 
Participation in public 
service (subsidized) 
employment 



Increiised parucipauon in 
training and employment 
programs 

Increased tmsubsidized 
employment 
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factors," but which did not materially alter ES* mission while introducing provisions on 
performance incentives: the dominant mission for labor exchange is employment, not 
enhanced earnings for the unemployed. The recently-implemented Food Stamp program 
similarly seeks the provision of services to assist "members of households participating ... 
in gaining skills, training or experience that will increase their ability to obtain regular 
employment" [Section 6(d)(4)(a)]. Other provisions in the legislation reinforce the short- 
run emphasis (e.g., "promptly"), as does the limited amount of funding appropriated for 
the program. All of these program goals are short-term and relatively clear . 

As a result of the 1986 amendments, the JTPA Summer Youth program has a 
broader set of purposes, although its focus is still largely short-run. In addition to 
providing summer work experience and exposure to the world of work for eligible youth, 
the program also seeks to enhance participants' basic educational skilU and to encourage 
school completion/enrollment [Section 251]. This represents the first explicit statement of 
purpose for the program. The program's goals are clear, and its time horizon has been 
stretched slightly. 

The second grouping consists of programs -- JTPA Title IIA and WIN — whose 
goals can be categorized as intemiediate-to-loneer term and which are relatively clear and 
unambiguous . JTPA's statement of purpose [Section 2] must be taken together with the 
more extensive provisions of Section 106 and explanations contained in the Conference 
Report to fully understand the purpose of Title IIA.^9 Section 106 explains that "job 
training is an investment in human capital not an expense," the productivity of which 
should be measured, for adults , by: 



Few programs have such clear direction. For youth , a number of goals less directly tied to 
outcomes (e.g., competency attainment, school completion, enrollment in other programs, 
among others) also apply. 

Since WIN's substantial restructuring in the *.arly 1980s, it is very difficult to 
characterize the programs at the national level in any meaningful terms. WIN is engaged in 
what amounts to a large, state-by-state demonstration of four broad programmatic 
approaches for eligible AFDC recipients and applicants. Yet, the purpose of WIN has not 
changed substantially from the pre-1981 period: restoring the families to independence and 
useful roles in their communities [Section 430]* This is to be achieved through: (1) 
employment in the regular economy; (2) training for work in the regular economy; and (3) 
participation in public service employment. The emphasis is on programmatic interventions 
leading to self-sufficiency through employment over time. WIN and JTPA Title IIA are 



-^Sce: Job Training Partnership Act: CONFERENCE REPORT . Washington. D.C.: U.S. House of 
Representatives, 97th Congress, 2nd Session, Report No. 97-889, September 28, 1982. 



Increased employment; 
Increased earnings; and 
Reductions in welfare dependency. 
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closely related programs both in terms of goals and time orientation, with WIN targeted on 
a subset of the JTPA-eligible population. 

The last group of programs - including JTPA Title III and postsecondary Voc Ed - 
exhibit considerable ambiguity in their goals and in their time horizon . There is no separate 
statement of purpose in JTPA for Title III. Neither the Act, the Conference Report, the 
regulations, or the numerous technical assistance guides (TAGs) contain a clear, concise 
statement of the program's goals. There appears to be consensus that the goal of the 
program is to re-employ displaced workers, many of whom are not economically 
disadvantaged, at earnings levels as close as possible to those prior to the plant layoff or 
shutdown. Whether this should be achieved through longer-term training or more rapid 
labor market re-entry efforts is not clear. Section 106 suggests performance measures 
(placement and retention) for Title III without clarifying its purpose. This absence of clear 
direction, coupled with the wide diversity of structures possible, has caused considerable 
confusion. Further, the program carries no legislative language concerning the time frame 
for interventions or results. 

The greatest ambiguity is in Voc Ed. The Act lists nine purposes for the program, a 
laundry list Vrith something for everyone which can be grouped for cwavenience as follows: 
improved target group access: improved skills acquisition/labor market responsiveness : and 
a catch-all miscellaneous category [Section 2]. The confusion stems from the fact that Voc 
Ed is less targeted than most of the other programs, has a long-run mission which has 
fluctuated between manpower training and education aims and is dominated by non-federal 
funding sources. In addition, while postsecondary Voc Ed is oriented towards serving 
those beyond high school and aiming for jobs not requiring a baccalaureate (i.e., 4-year) 
degree, it labors under a definition which encompasses preparation for both paid and 
unpaid employment [Section 521(31)] and which includes as one of its goals, improving 
the effectiveness of consumer and homemaking education. This, coupled with the fact that 
this muddled mission applies only to the federallv-funded portion of Voc Ed - less than 10 
percent of the total nationally - makes the going very tough. If the Act is not clear on just 
where and when the program should be going, it is very difficult to envision appropriate 
measures and management approaches to determine whether it has arrived. 

B • Level of Use 

The actors involved in pcrfoi lance management decisions vary and the nature of 
their information needs differ as well between programs. There are program goals and 
objectives at the federal, state and local levels with varying degrees of specificity, requiring 
different standards and measures at each level. Standards should fit the management 
functions and decisions required at each level. Differences between programs are related in 
part to variations in the share of federal funding. 

National-Level Policymakers, Two actors have a primary interest in 
performance standards at the lational level. First, Congress authorizes programs and 
appropriates most of their funding. It follows that Congress has an interest in the 
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performance of these various programs measured against some type of benchmark. This is 
the basic Congressional oversight function. The analogous incentives and sanctions are 
found in changes in the level of appropriations and in authorizing decisions over time. 
Second, federal agencies also have a keen interest in program performance in keeping with 
their role as members of the Executive Branch. In many cases, this responsibility is 
legislatively mandated; in others, it is not. Furthermore, for some programs, federal 
commissions - such as the National Commission for Employment Policy (NCEP) are 
charged with evaluating programs a.id advising Congress, the President, and the agencies 
on ways to improve them. 

At issue is the type of measure appropriate for national-level use. Congress and the 
federal agencies are broad policymaking bodies and should focus more upon longer-term, 
net impact - evaluative - measures of performance. Neither are responsible for day-to-day 
management of service delivery for participants; local entities, and to a lesser extent, states 
are. The federal role in human resource management should be establishing the overall 
program goals and objectives and creating an environment within which good state and 
local program management towards those ends can occur. 

State-Level Policy and Program Officials, State management 
responsibility for human resource development has increased considerably in the 1980s. 
Governors or state agencies now typically have far more discretion - from front-end 
planning and resource allocation to program oversight - in JTPA, ES and welfare 
programs than they did just a few years ago. The need for perfonnance measures at the 
state-level has increased accordingly. Governors and their agencies must oversee the 
programs and, in many instances, financially reward and/or sanction local program 
providers based on their performance. The nature of the standards required at the state 
level, in terms of detail, frequency, and other factors, rests somewhere between those at the 
national and those at the local level. 

With the ac'ied attention to state performance management across programs, the 
importance of inter-program coordination and the rationalization of delivery systems have 
taken on greater importance at this level as well. Some of these programs contain 
provisions authorizing the establishment of siate coordinating councils of differing makeup, 
typically appointed and serving at the pleasure of the governor, designed to carry out these 
responsibilities. These councils may be charged with examining program results in several 
related programs and advising the governor or his agency on performance standards and 
how best to rationalize service delivery in the state. The potential role of these councils is 
great but not yet fully realized. 

Another issue which has received little attention in the discussions of management 
responsibilities and performance standards to date is whether the governors - or any other 
elected official at any level, or appointees serving "at the pleasure of - are appropriately 
charged with carrying out management functions. Governors (and elected officials 
generally) are necessarily politicians responding to political demands, typically more 
concerned with distributional than results-oriented objectives. In none of the programs 
examined here do governors have anything to gain from dutifully performing these duties: 
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not only will their states continue to receive the same funding levels after doing so, but they 
will have alienated potential allies in the process. Put another way, governors can only 
make enemies in carrying out these duties at the substate level. The only benefits lo the 
state (e.g., a few more citizens/voters served better at the margin) are likely to be realized 
over a time period likely to exceed their tern in office. 

Local Officials and Service Providers, Local actors have the most 
immediate need for perfonrinue measures and standards. Increasingly, the responsibility 
for performance management is shared by three sets of actors: local elected officials, the 
private sector, and service providers. Locally, data concerning performance measures must 
be detailed, reliable and available with greater frequency (i.e., daily not quarterly). Day-to- 
day management reality dictates that local managers make do with the best proxies they can 
afford, trading off ideal measurement devices in favor of pressing short-term management 
requirements. Among these programs, there is remarkable diversity in the entities 
responsible for local management and service delivery. Some are autonomous, 
incorporated organizations with relatively strong private sector feature, others are 
independent locally-elected boards, while still others are wholly-owned units of the state 
agency in effect. The local management context - much of which is driven by both the 
enabling legislation and the relative federal/state/local funding contributions - has important 
implications for the requisite systems and standards. 

JTPA, Nationally, both Congress and the Executive Branch play important roles 
in ensuring JTPA program performance. (Figure 3 summarizes the roles played by the key 
actors in each of the programs.) Congress' role is intermittent and more distant than that of 
the Department of Labor. It is concerned with performance in a broader evaluation context 
and oversees Labor's implementation of JTPA. The Secretary has broad evaluation 
responsibilities as well, but must also set performariv . standards f'"'* adults and youth in 
Title IIA, al )ng with the framework ('"parameters") within which governors may vary the 
SDAs' standards. The Secretary requires both longer-term net impact measures to fulfill 
evaluation needs, as well as more immediate proxies. The Secretary's needs in terms of 
performance management extend only to what is necessary to proviue governors the 
standards and accompanyir.3 framework within which to oversee SDA program delivery. 
Tlie latter function requires detailed data collection and reporting to yield the data base from 
v/hich to develop the standards. 

The NCEP, has been charged with identifying employment and training needs, as 
well as evaluating the effectiveness of Federally assisted programs Zi has explicit 
legislative responsibility for advising the Secretary of Labor on JTPA performance 
standards and their effectiveness. 

At the state level, governors not only establish state program goals and objectives, 
but also develop and implement state policy on performance .standards, with the advice of 
the state job training coordinating councils (SJTCCs). While removed from day-to-day 
program management, they are far more involved than before and require sufficiently 
detailed data to determine whether an SDA is performing well against its standards and why 
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Key Program 



TTPA 

National Secretary of Labor 

• Develops performance standards 

• Sets parameters for governors 
to vary SDA standards 

• Rules on SDA sanctions appeals 

National Commission for 
Employment Policy 

• Reviews JTPA effectiveness & 
coordination with other programs 

• Advises Secretary on standards 



State Governor 

• Establishes state goals/objectives 

• Prescribes variations in standards 

• Establishes & applies policies for: 

- Distributing SDA incentives 

- Providing SDA technical 
assistance 

- Reorganizing SD As 
(corrective action) 

• Establishes added standards 
(optional) 

Suate Job Training Council 

• Develops Governor's plan 

• Recommends standards variations 

• Advises on SDA plan approval 

• Reviews SDA performance 

Local PICs/SDAs 

• Plan & provide for 
cervices to exceed standards 

• Develop/recognize youth 
employment competencies 

• Negotiate performance contacts 



ESl 

Secretary of Labor 

• Develops standards 
(optional) accounting 
for differing state 
priorities (required) 

National Commission for 
Employment Policy 

• Reviews ES effectiveness 
& coordination with other 
programs 

• Advises Secretary 0.1 
standards 

Governor 

• Adjusts statc/substate 
standards 

• Establishes/applies 
policies for: 

- Incentives disuibution 

- Technical assistance 
(optional) 

- Corrective action 
(optional) 

• Reviews state ES plan 
State Job Training Council 

• Advises governor on 
local plan approval 

• Reviews local ES 
performance 

ES Offices 

• Plan (jointly 

with PICs/LEOs) & 
provide services to exceed 
standards 



Q ^ Postsecondary 
£ RJ^C E"^Ploy"^cnt & training portion 



Figure 3 

Actors and Their Roles 



VOC ED* 

Secretary of Education 

• State plan approval 

• RD&E 



National Council on Voc Ed 
• Auvises Secretary on Voc 
Ed effectiveness and 
coordination with other 
programs 



State Board of Education 

• Receives federal funds & 
determines sccondary/post- 
secondary allocation 

• Establish criteria for Voc 
Ed evaluation & assess 
access to quality program 

• Evaluate ^equacy& 
effectiveness of Voc Ed & 
J' PA and coordination 

State Advisory Councils 

• Advise State Board on 
evaluation criteria, etc. 



WIN 

Secretary of Health & 
Human Services 

• Determines planning and 
operations parameters 

• RD&E 

[Joindy with Sec. of Labor 
for regular WIN program) 

National Commission for 
Employment Policy 

• Reviews WIN effectiveness 
& coordination with 
other programs 

Governor; .welfare agency 

• Chooses program options 
(WIN, WIN Demo, etc.) 

• Plans service delivery & 
establishes goals and 
objectives 

• May contract with providers 
(e.g., ES) directly 

State Job Training Council 

• May serve as WIW state 
council 



FOOD STAMPS* * 

Secretary of Agriculture 

• Establishes national participa- 
tion standards and key exemp- 
tions 

• Determines program planning 
& operations parameters 

National Commission for 
Employment Policy 

• Reviews E & T program 
effectiveness & coordination 
with other programs 

Governor/welfare agency 

• Plans programs and 
establishes goals 
and objectives 

• Establishes additional 
panicipation exemptions 

• May contract with providers 
dirccdy 



Local Boards 

• May oversee 
planning & operations 

Community Colleges, Tech- 
nical Institutes, Proprietary 
Schools 

• Plan & provide services 



Welfare Office 

• Coordinate service 
delivery 

• May contract with 
providers 



Welfare Offices 

• Coordinate service 
delivery 

• May contract with providers 



if not. The need for knowledge about non-performance stems from the governors* 
legislative mandate to provide technical assistance to SDAs and to sanction those not 
meeting standards. Their technical assistance/sanctions role calls for greater detail, 
frequency and access to SDA data than the incentives role alone. Governors need detailed, 
frequent reporting on performance, process- and outcome-oriented, to fulfill their expanded 
planning and controlling roles. The burden for the cuixent performance standards system is 
primarily at the state-level for Title IIA and even more so for Title III. 

The potential impact of the SJTCCs is large. These councils not only serve a major 
advisory role in JTPA performance management as described above, but they also carry out 
key review functions for the states' ES, Voc Ed and any other plans providing 
''employment, training, and related services" in the state [Section 122(b)(8)]. Moreover, 
governors have the discretion to transfer related coordination functions assigned to L'ke 
committees or councils under WIN and ES to the JTPA councils. This latter provisioji 
opens the door to much closer coordination among the many employment and training 
programs, presumably including/ the essential performance management aspects. 
Consolidating oversight and performance management functions - including advising 
governor., on planning parameters, duplication of services, program evaluation and 
standards and incentives/sanctions policies~in the SJTCCs has the potential for 
rationalizing the broader HRD system to a fairly great extent. Of course, as explained 
below, some of the other programs (e.g., Voc Ed) have their own advisory councils aM 
very different funding and structures which might present barriers to fully pursuing this 
process. 

There is a major baixier to effective state implementation of performance 
management piovisions that needs to be acknowledged. As presently designed, governors 
have little to gain from doing so. Governors who spread incentive funds widely, treating 
most SDAs as high performers, and administer pats to the wrists of chronically 
underperforming SDAs and governors who limit incentives to excellent SDAs and 
reorganize poor performers are treated alike: both receive the same JTPA funding 
levels/shares for next year regardless since Title IIA funding for states is based on 
measures of need. It is unclear why they jjhould bother about performance management at 
all. They can make only political enemiei* of SDAs sanctioned in the process with nothing 
else to show for it. Of course, for Title III there is absolutely no reason to do more than set 
perfunctory goals or standards. Neither the governors nor the local providers have 
anything at stake. JTPA Title III funds also are allocated to states solely on the basis of 
need. Moreover, the funding is 100 percent federal; states, with few exceptions, are only 
working \^ith federal funds in any event and have little of their own funding at stake. 

For the PIC and for the SDA director. Title IIA (and possibly III) standards 
represent only the barest of performance measures required for managing service delivery 
at the local level. They will want to have monthly, weekly and even daily performance 
reports for an expanded set of measures which may contain breakdowns by age, race, sex 
and program activity/service, as well as by service provider. For youth, the various types 
of competencies - pre-employment, work maturity, skills, etc. - and possibly their 
elements will need to be monitored. Given the standards adjustment methodology (the 
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DOL models) used by most states, SD^ "irectors will probably also want to track closely 
participation levels and changes in the v. jus external factors contained in the model.30 

ES. Like JTPA, both the Congress and the Department of Labor have an interest 
in ES performance nationally, with USES being the primary federal actor for the Secretary. 
ES performance-related provisions are more discretionary than under JTPA, allowing 
rather than mandating the Secretary' to establish standards (Figure 3). The legislation does 
require the USES Director to establish Federal Advisory Councils [Section 11], but their 
management role is unclear. The federal role is relatively weak in labor exchange 
performance management, despite having 100 percent federal funding.31 

At the state level, the actors' roles have been modified significantly. Before 1982, 
labor exchange funds were allocated from USES directly to SESAs. Governors may or 
may not have played a significant role, depending on the state's enabling legislation. With 
the 1986 JTPA amendments, funds now pass through the governors with 10 percent is 
reserved for their discretion. (Nearly all ren percent funding ultimately must end up in the 
hands of the local ES offices.) Experience with governors' discretionary funds varies 
viridely. Some pass it along to their SESAs with no strings attached and no further 
direction as to its special uses, while others make full use of this new-found discretion to 
shape policies and programs, including Implementing incentives policies for local offices 
[Section 7(b)(l)].32 The legislation aI:o mandates the creation of state advisory councils to 
parallel those at the federal level, although governors have the option of using their JTPA 
state councils for this purpose [JTPA, Section 122(b)(c)]; how many do so is unknown. 
ES state councils are not accorded explicit standards and ii.centives responsibilities that 
their counterparts have under JTPA. 

The same barriers to implementing standards and incentives for JTPA prevail for 
ES as well at the state-level. With federal funding allocated exclusively cn basir, of 
need (i.e., shares of the civilian labor force and unemployment), the rationale foi ^^ovemors 
or state ES administrators to pursue syst-.ms for rewarding and sanctioning locafoffices is 
lacking. In earlier periods when the federal-stare ES allocation fomulas were based in part 
on prior performance, these new incentives provisions might have fi^ 'xtter. 

Locally, labor exchange is a very different system. While JTPA is highly 
decentralized with widespread variation, local ES offices are typically wholly-owned and 



30Thcre arc several models of local service delivery, one in which the PIC acts a?, a resource allocation 
enuty subcontracting entirely to providers, another in which it acts as a self-contained delivery agent, and 
yet another which is a combination of the two. The first and last models appear to be mor- common. 
31 This may stem from the unique nature of these "federal funds": through a complex fede"|/state funding 
arrangement, the funding to support the national labor exchange system is deiived largely fron taxes on 
employers passed through to the federal government wi(h state credits and chargebacks. In effect, labor 
exchange funds are state-levied employer tax revenues fiLc. Jd up thiough the Congressional appropnauons 
process and returned to the states. 

^^Interestingly, Cook and Fisher (1986) discuss the uses of the 10 percent funds without mentioning 
whether the governors were directly involved. They indicate that more than a third of states were using 
these funds for performance incentives, a figure which seems high. 
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operated subsidiaries of the state ES, to use a private sector analogy. Local ES staff are 
state employees providing services at the local level, similar tc ocal welfare agency staff. 
However, with the JTPA amendments,. ES planning is conducted jointly with SDA staff. 
The PICs are inserted into an ES oversight function locally as well. 

Welfare Employment Projjrams. There is considerably more variation in the 
performance management actors at most levels for vvelfare employment and training 
programs than for the other efforts. And, while most of the funding is federal, the share of 
the federal contribution differs by program with significant implications for standards and 
incentives policies. 

At the national level, the major actor for both WIN options is the Secretary of 
Healih and Human Services (HHS), however, for nearly half of the states with regular 
WIN programs, both Labor and HHS are involved. In the ^ood Stamp programs, the 
Secretary of Agriculture is the top federal actor in the management hierarchy: the Secretary 
establishes national participation standards for Food Stamp programs. Base Food Stamp 
funding is federal with a one-for-one federal match entitlement above that level for any state 
funds expended; WIN is 90 peicent federally funded. Possibly more important than the 
share of employment and training program funding is the share of regular cash assistance 
support. Food Stamps is exclusively a federally-funded assistance program, while the 
federal share in financing AFDC benefits varies between a floor of 50 percent and roughly 
78 percent depending on each state s per capita income; the costs of administration for these 
programs are shared 50/50. States committing more of their own limited funds will 
obviously have more at stake in operating effective programs, since they stand to reap 
pc^ential benefit financially. These states would be more likely to take management and 
performance standards seriously. 

At the state level, governors now have much more discretion than at any time since 
the federal/state welfare system was first established more than fifty years ago in devising 
and implementing welfare employment and training program initiadves. Under WIN, 
governors - typically acting through state welfare agency boards - have an array of broad 
program options to choose from as discussed. The WIN agency also serves as the state 
administering entity for Food Stamps and related welfare employment and training 
programs. Despite the inclu^on of legislative provisions encouraging closer coordination 
between WIN and programs such as ES and JTPA, the mechanisms to promote it are 
lac" g, as dst rewards for doing so at the state level. Much like the other programs, there 
are few real incentives for managing towards high performance in these efforts, unless the 
potential budget savings - of state general revenue in those states which have sufficiently 
large state contributions leads in this direction. A compliance orie^itation is likely to 
characterize •'am manogement. It is no accident that state welfare i^valuation ui.its are 
focused upo .rDC benefit payment error rates: excessive error rates (payments to 
ineligibles a^ . ptiyment of too much in benefits to those eligible) result in reduced federal 
funding for rl.^. stme's AFDC program. 

Locally, both WIN and Food Stamp programs are operat^^d by local offices of the 
state welfare agency, although the actual provision of services may occur through a mix of 
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contractors that varies greatly from area to area. In some areas, the local ES may be the 
sole provider, while in others the list of deliverers may range fiom the PIC and ES to 
community colleges and proprietary schools. The responsible entity is the local office of 
the state welfare agency as a rule. Since the early 1980s, it has become very difficult to 
characterize local programs and their key actors. Little structured information is available 
on a regular basis in this regard,33 

Postseconrfary Voc Ed. Despite the fact that Voc Ed is one of the major 
providers of job training services nationally for JTPA participants, AFDC recipients and 
refugees, the nature of its system - from its purpose, goals and objectives to its funding, 
its actors and their responsibilities — diff^^rs markedly from the others. While 
postsecondary Voc Ed too is a federal grant-in-aid program, the amount of federal aid in 
relation to state and local funds is so small as to alter the character of the system 
dramatically: with only about 6-10 percent federal funding, federal performance provisions 
or expectations laid on state and local recipients must be implemented via suasion more than 
mandate. Influence and bargaining, not control, are the essential strategies in Voc Ed 
performance management. 

At the federal level, the key actor is the Secretary of Education. The Secretary is 
advised by the National Council on Vocational Education (NCVE) cn numerous aspects of 
the programs, including the "effectiveness" of the Perkins Act and its implementation, and 
coordination with JTPA [Section 431]. The NTEP also is charged with examining the 
effectiveness of Voc Ed programs and reviewing the NCVE reports, but the roles of each 
of these two commissions are vague in relation to the other. There is slight membership 
overlap: the chairman of the NCEP serves on th^ National Council on Vocational 
Education. 

In each state, the board of education rather than the govemor^s the recipient of all 
Voc Ed funds and further determines the within-state split between secondary and 
postsecondary programming. In some states these boards are appointed by and serve at the 
pleasure of the governor, but in others they are elected and enjoy broad, longstanding 
autonomy. They are advised by state councils ~ the state-level counterparts to NCVE - on 
such tasks as establishing the criteria for evaluation and assessing access to quality state 
Voc Ed programs [Section 112]. They also have the responsibility for evaluating the 
adequacy and effectiveness of both Voc Ed and JTPA and their coordination at least once 
every two yeai'S. 

Locally, there are community colleges, junior colleges, proprietary schools and 
technical institutes, not to mention local education boards to consider, many of whom 
operate with near autonomy from the others as well as from the state education entity.^^ 
There are instances, in which local colleges have been engaged in operating worldwide 
postsecondary Voc Ed programs, funded in part by the military, with many more students 
outside their distiict than in it. MIS requirements and accounting have typically been much 



33For example, see: Nightengale and Burbridge (1987), p. 29. 
3^Sec: Hoachlandcr, ct. al. (1985). 
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looser than in the other HRD programs. Such a situation makes any notions of control 
highly problematical at best. 

C. The Role of Performance Standards 

Management, whether public or private, should perform the following basic 
functions: planning, organizing, staffing, directing, and controlling.^^ Performance 
standards support the control function, enabling management to determine whether goals 
and objectives are being met. But, standards can be legislated and implemented for a 
number of reasons, depending upon management's operating philosophy. Management 
pursuing superior performance across the board would design and implement standards in 
one fashion. Management adhering to a traditional management-by-exception approach 
would want to devise standards :o allow effective separation of high, medium and low 
performers, facilitating more efficient allocation of management efforts. Performance 
standards should be established to promote fulfillment of the program*s goals and 
objectives, such that managing towards the standards results in goal attainment. Standards 
translate program goals and objectives into practical performance expectations. On a day- 
to-day basis, these are translated into numerical targets in the providers* contracts. Only 
three programs contain legislative and regulatory provisions dictating the establishment and 
intended use of performance standards as such: JTPA, ES and Food Stamps. (Figure 4 
summarizes these provisions.) The others address the role of standards in various ways, 
some mandating measures of effectiveness and others ignoring them altogether* 

JTPA. JTPA Title IIA performance standards for adult? and youth were 
authorized to restore the program*s credibility, by ensuring that, unlike CETA, JTPA 
would be performance-driven. Standards were also to support broader evaluation needs, 
measuring the extent to which programs were achieving their goals, nationally and at other 
levels as well, over the longer term. Finally, standards were to provide state and local 
entities with the monitoring tools needed to manage the programs effectively on a day-to- 
day basis. This last function is spelled out most clearly in the TAGs disseminated by the 
Depai-tment of Labor to states and localities. Congress intended standards to be the tool 
enabling the Secretary of Labor and the governors to push job training programs - with 
cash rewards as the carrot and technical assistance, along v^ith the threat of reorganization 
as the stick - to attain results defined in terms of outcomes during the program (for youth), 
at the point of termination and in some post-program period. Standards were to lead to 
improved longer-term results for investments in training. Corresponding attention is given 
to cost measurement as well, suggesting that standards have a two-fold purpose: improved 
effectiveness (better results) and enhanced efficiency (tighter cost management). 

Overall, the performance management system envisioned in JTPA is more thorough 
and contains far less ambiguity than any of the other programs. But, there has been some 
confusion concerning the management philosophy underiying the standards. Until 
recently, the incentives and sanctions structure tied to the standards established an either/or 



^^For discussion of management functions and the role of performance standards, sec. King and Geraci 
(1983). For a more generic discussion of control systems in public programs, see: Williams (1980). 
Chapter V. 
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Figure 4 

Major Performance Standards Provisions 



JlEA 



FOOD STAMPS ^ 



Standards 



Title IIA 

• Established by Secretary ,varialions 
allowed by Governors, as follows: 

Adult measures may include: 

- placement retention unsubsidized 
employment; 

- increascin earnings/wages; and 

" reduction in welfarccceipt/amounL 
Added Youth factors shall include: 

- attainment of PIC employment 
competencies; 

- school completion or equivalent; 

- enrollment in training/apprentice- 
ship, or enlistment in Armed Forces. 

• Postprogram and gross program 
expendiiure measures [106 (b)] 

• Standardsonly modified every 2 years 
[106(d)] 

• Standards may vary based on [106 (e)]: 

- economic, geographic, & demographic 
factors; 

- service population; 

- service mix. 



Title III 

• Established by Secretary 
based on [106(g)]: 

- placement and 

- retention in unsubsidized 
employment. 



Secretary may establish 
standards that vary with 
State plan priorities 
[13(a)]. 



Secretary must establish 
participation standards for 
non -exempt Food Stamp 
recipients [6(d)]. 
Standards may vary based 
on: 

- characteristics of 
persons served; 

- types of nrograms 
offered 



Incentives & 
Sanctions 
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Governor provides [106 (h), 202 (b)(3)]: 

- incentives to SDAs exceeding standards; 

- technical assistance to all SDAs. 
Governor must reorganize SDAs for 
failure to meet standards [106 (h)]. 



Governor may use 10 per- 
cent discretionary funds for 
for incentives for ES offices 
and programs [7 (b)]; 
Incentives account for: 

- direct/indirect placements: 

- wages on entered employment; 

- retention; or 

- other factors. 



Employment & training program portion. 



Secretary may withhold 
FoodSiampE&Tfunds 
for state failure to meet 
standards. 
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situation: an SDA was either a wonner by exceeding its standards or a loser by only 
meeting or failing to meet them [Section 202(b). Winners were to receive cash incentives; 
losers were to be provided technical assistance and/or be reorganized if the situation 
persisted for two consecutive years. This confusion has been ameliorated slighUy by the 
1986 amendments which pem^ programmatic technical assistance for aJl SDAs regardless 
of performance. This post- l98a structure would support a management approach geared to 
leading the entire system towards high performance, holding out incentives for the high 
performers and assisting all to improve over time. 

In the process of implementing these standards, there has been a significant, but 
little noticed, shift in federal policy. The initial emphasis was clearly on productivity 
improvement, pushing the system to higher performance over time. The "departure 
points," the levels from which SDAs* performance expectations were derived, in the 
Department of Labor's Performance Standards Adjustment Model were set at levels which 
represented average performance in the system. SDAs performing well above their 
standard were viewed as succeeding, while those who fell beiow were not. However, 
since PY 1986, departure points have been set with the expectation that fully 75 percent of 
the SDAs wouiu exceed the standurds.36 The underlying rationale has sh'fted so that 
standards now represent "minimallv acceptable levels of performance " for most measures. 
Credibility may suffer over time as the job training system and its ooservors come to 
understand that many of the so-called winners actually posted below-average performance. 

The intended uses of Title III perfonnance standards are unclear at best. Without a 
well-defined program mission, it is easy to understand the difficulty involved with giving 
clear definition to the role which standards are to play. Title III standards appeal- to have 
been added as an afterthought. Section 106 of the Act directs the Secretary to establish 
Title III standards based upon placement and retention but Joes not explain what they are to 
be used for. The administrative structure of Title III is less proscribed than for Title II A as 
v^cil. The Secretary must allocate three-quarters of the funding to the states based on a;i 
allocation fonnula, and • ^sponsibility for planning and operating dislocated worker 
programs res.s with the governor. The governor must consult with PICs (and others 
including labor organizations) but is not required to rely on existing Title IIA SDAs for the 
delivery system. Unlike T'\t IIA, Title III is inherently an ad hoc program focused largely 
on immediate events sucii as layoffs and plant closings. There are no rew? cr sanctions 
tied to local program performance. It is worth pointing out that there is little lOr governors 
to gain in rewarding or sanctioning local prc^^rams based upon performance, since in many 
cases, the situation which called forth the Title III funding will be a one-time occurrence. It 
is not ac all clear what role standards should play in Title III. 

For Title IIB Summer Youth programs, there are no standards per se > only local 
goals now prepared by SDAs with state guidance. Yet, these programs have been regular, 
on-going programs, even more so since the 1986 amendments allowing for closer 
integration of the Title IIA and IIB efforts and operation of year-round IIB programs. With 



36For an explanation of this change, sec: West and Dickinson (1986). 
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these recent changes, there appears to be a clearer rationale for the initiation and use of 
standards with a role similar to the regular IIA efforts. 

ES, The intent behind and role for ES standards is less clear than under JTPA Title 
IIA. The Secretary of Labor is authorized but not required to establish performance 
standards [Section 13(a)]; and, if established, those standards must "... take into account 
the differences in priorities reflected in State plans." As amended in 1982, the Act 
envisions state-level standards, possibly established by the Secretary, tailored to 
differences in state priorities. Because of the unique financing mechanism for ES, state 
labor exchange allocations are essentially state tax revenues which are being returned after 
having been temporarily received by the feds and appropriated. Labor exchange does not 
qualify as a federal/state grant-in-aid program the same extent as some of the others, and 
this affects the expected role of performance standards.^^ 

Two significant changes were legislated in 1982, pointing in different directions. 
First, the formula for allocating ES funds to the states was changed to a needs basis , 
allocating funds based on relative shares of the civilian labor force and unemployment 
(Section 6(b)]. Performance no longer plays a role in the amount of funding a state 
receives. Second, a new.Seccion 7(b) creates a discretionary fund , 10 percent of a state's 
total allocation, to be utilized by the governor for, among other uses: 

• "performance incentives for public employment service offices and 
programs, consistent with performance standards established by the 
Secretary, taking in^o account direct or indirect placements wages on 
entered employment, retention, and other appropriate factors." 

Congress apparently felt some of the same concerns about the ES that it felt about 
the job training system - particulariy, that it needed to be performance-driven to some 
extent, and that governors should play a greater role in the process. However, whether 
from the complexity of the ES mission or from a weaker sense of need for perfoiTnance 
elements in such an open-entry system, the role for standards and the fiamework for 
performance management was left undefined. Some of the measures suggested (e.g., 
wage at placement and retention) go well beyond the short-term exchange function and hint 
at longer-term impacts. Wide discretion in ES performance management has been created 
for governors but little has been used.38 

The Department of Labor has proceeded much more cautiously in implementing 
standards than with JTPA. Despite cleai legislative encouragement for joint state and local 
ES/JTPA planning and concern over performance, the Department has adopted a relatively 
passive role. There are brief statements in the TAGs concerning the uses of standards at 
the substate level for manaeement and technical assistance and evaluation and monitoring . 



^^Labor exchange services are financed with federal and stale uncrriploymcnr ^^xes through an claburaic "lax 
offset" system, making ES less than a real grant-in-aid program. 
38cookand Fisher (1986). 
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but little has actually been done since the optional standards adjustment models were 
developed in the early 1980s. 

Food Stamps- The 1985 legislation authorizing Food Stamp employment and 
training programs contains specific language governing the establishment and use of 
participation standards [Section 6(d)(4)(J)]. The Secretary of Agriculture is required to 
establish state standards for the percentage of non-exempt persons receiving food stamps 
who will participate in the programs. Standards may vary by state depending upon the 
characteristics of the persons required to participate and the types of programs to which 
they are applied. Ultimately, the Secretary is authorized to withhold federal employment 
and training funds for state failure to meet any such standard [Section 6(d)(4)(K)]. 
Additionally, the Secretary is to monitor the effectiveness of the programs "... in terms of 
the increase in the numbers of household members who obtain employment and the 
numbers of such members who retain such employment as a result of their participation in 
such employment and training programs [Section 16(h)(5)]." Thus, Food Stamp standards 
differ markedly from those in JTPA and ES: they were designed and implemented to 
increase participation in short-term programmatic treatments to move households (not 
already required to register for services under WIN) off the Food Stamp rolls. While there 
has been some discussion of instituting other types of standards (i.e., outcomes), no action 
has been taken in this regard.39 



None of the other programs contain results- or outcome-oriented standards. The 
Perkins Act refers to "evaluation criteria" [Section 112] and the establishment of "measures 
of effectiveness" [Section 113], not standards. Nearly all the programs - from JTPA and 
ES to WIN and Voc Ed - contain language concerning participation levels. WIN's 
provisions are in tems of groups required (or exempted) to register for services, not in 
terms of any ^. . -set level or proportion that must participate. To date, most programs have 
not responded to the issue of performance standards' role. Only JTPA has adequately 
addressed the role performance standards should play in program management. As 
mentioned above, the role should tie directly to the mission, goals and objectives, and it 
should be tailored to the particular programmatic context. For example, while outcome- 
based standards need to be established for Voc Ed, given the legislative and programmatic 
environment, it is not clear either that their role could parallel that for JTPA or that, if it did, 
they would be effective. 

D . Appropriate Measures of Performance 

A fourth major issue is what are appropriate measures of performance. There are 
numerous dimensions to thi.s issue.^^ The first two relate to the degree to which the 
measures are related to the objective in question; the others are more concerned with how 
well the measures capture what is inten^^d. 



39For example, see: Gu'^ron and Sherwood (1986), p. 82. 

^^For an excellent discussion of these and related issues in the context of job training programs, sec: 
Sum, Andrew, ct. al. (1978). 
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Process v. Outcome Measures. A distinction is typically made between 
process and outcome measures .41 The former include in-program measures, - xh as costs 
(aggregate or per participant), panicipation rates/levels for various groups, ana activities or 
services received, among others. Outcome-based measures are tied to at-tenninatin.i or 
EfiSI-program events, such as job placement, employment retention, increased wages or 
earnings, reductions in welfare grants or case closures, and others. Most of the attention 
here is on outcome rather than process standards, although to the extent that programs have 
process-oriented objectives, standards should reflect them. 

Efficiency v. Effectiveness Measures. Measures can also be classified as 
efficiency- or effectiveness-oriented, or some combination of the two. Efficiency measures 
capture operational "tightness" at a point-in-time and include cost-per-panicipant and cost 
per-entered employment standards, service units per staff year, case closures per staff year 
and similar measures. Effectiveness measures emphasize outcomes regardless of cost or 
level of inputs (e.g., placements and placement rates, wages at placement, employment 
retention). It is possible to operate an effective program inefficiently; conversely, it is 
possible to run an ineffective program efficiently. Some balance between the two is 
desirable, since too much emphasis on one may preclude attaining the other. Cost- 
effectiveness measures represent a combination approach. Measures such as cost per 
earnings or employment gain fall into the cost-effectiveness category and tend to be used 
more often in program evaluation. 

Proxy Measures. Especially for programs designed for longer-term impacts, 
there is a question about the use of p.oxies, near-term indicators of the actual result 
desired. -'^ if the desired outcome is increased long-term earnings or reduced welfare 
dependency, then measuring that impact will clearly take many years. Given that 
performance standards must provide useful information for management, waiting for the 
ultimate impact measurement may preclude having any effect on immediate outcomes' 
There is also a question concerning just how close a proxy has to be to the true outcome to 
constitute an acceptable proxy. For longer-term impacts, it is axiomatic that the longer the 
measurement period, the better the proxy. A 3-year earnings average will better 
approximate an individual's lifetime earnings than one based on a 3-month, post-program 
period. But how closely coirelated does the proxy need to be to qualify as a good proxy? 

A related issue concerns the use of net versus gross impact measures. Net impact 
measures are prefeired to gross impact measures in a programmatic world with no practical 
constraints. That is. it is preferable to have a measure which refiects the actual contribution 
of th.- program intervention to the participant's success, rather than the combined impact of 
the treatment, the individual's characteristics, local environmental factors, and other non- 
treatment factors. However, the use of net impact measures for standards is not feasible. 



Williams (1980) groups measures into distributional, proximate and final outcomes, where distributional 
outcomes are basically in-program outcomes. There are other classifications loo numerous to mention. 
Mmong other sources, sec: Borus (1979): Gay and Borus (1980); Geraci (1984); Kogan. el. al. (1985); 
and Zomiisky, el. al. (1985). 



35 



There are time period and methodological problems, not to mention operational constraints. 
Net impact measurement is preferred for evaluation, not for near-term management.'*'^ 

Other Dimensions, Measurement accuracy entails both validity and 
reliabiliiy.'*'^ To be valid, a measure must capture what it is both intended and presumed to 
measure. Validity encompasses the consistency or fit of the measure in relation to program 
goals and objectives and the extent to which it reflects what others think it should. To be 
reliable, a measure must yield the same results with repeated measurement under similar 
conditions. Accurate measures possess both attributes. From a management perspective, 
this suggests the need for serious attention to consistent data collect jn - quality control - 
and periodic validation. 

JTPA. Title IIA has relied upon the same seven performance standards for adults 
and youth since its inception, but the standards' numerical levels and the models for 
estimating and adjusting them have changed somewhat from year to year. Major changes 
in the standards are being instituted on July 1, 1988 (Figure 5). Both the exi'^ting and 
prospective standards for Title IIA are predominantly outcome-oriented , effectiveness 
standards . Most were selected because they proxy to some degree for longer-term, net 
imp?,CiS of the orogram, whether employment and earnings increases or reductions in 
welfare dependency.'^S Only the cost and youth measures have a noticeable process 
orientation. Ev^u the youth positive termination and employability enhancement rates 
measure intermediate outcomes related to ultimately securing employment.^^^ The post- 
program measures are decidedly outcome-oiiented, shifting the focus of the job training 
system towiu-ds long-run return on investment. The Department has expressed the desire to 
push for quantity and quality JTPA outcomes, in what can be termed an *'8-l-l Rule." In 
the eight out of twelve .standards which must be factored into theii incentives and sanctions 
calculations, governcr.<v must use at least one "quality-of placement" standard - wage at 
placement or post-program earnings. Additionally, either the youth positive termination or 
employability enhancv^ment rate must alSD be utilized. 

With the exception of the two cost standards, the remaining Title IIA standards are 
effectiveness standards. The cost per entered employment and cost per positive termination 
standards balance the effectiveness standards, ensuring that entered employment rates are 
achieved at a reasonable cost. These two are combination, cost-effectiveness standards. 
The "8-1-1 Rule" provides considerable policy and administrative discretion at the state and 
local level, while simultaneously encouraging overall attainment of federally-desired 



43For an excellent series on state and local evaluation, sec; Jl ?A Evaluation Design Project (1986). 
^^Rossi and Freeman (1982) contains a concise discussion of mcosuremenl accuracy, pp. I87ff. 
45For a sampling of this research, see: Kogan, et. al. (1985): Dickinson and V^'cst (1985); Geraci (1984); 
and Zomiksky, ei. ah, (1985). 

^^positive terminations include entered employment, as well as attainment of a youth employability 
enhancement; youth employability enhancements include attainment of a PlC-recognized youth 
employment competency, entering non-Title 11 training, returning to full-lime school, completion of major 
level of education, and completion of program objectives (for 14-15 year oKls). 
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Current 



JTPArPY1988-89> 
Title IIA 



ADULTS: 

Entered Employment Rate 68 % 

Cost per Entered Employment $4500 

Average Woge at Placement $4.9j 

Welfare Entered Employment Rate 56 % 

YOUTH: 

Entered Employment Rate 45 % 

Positive Termination Rate 75 % 

Cost per Positive Termination 54900 

♦Employability Enhancement Rate 30 % 

POSTPROGRAM (Adults only]: 

*Follow-Up Employment Rate 60 % 

♦Welfare Follow-Up Employ. Rate 50 % 

♦Weekly Earnings at Follow-Up $ 1 77 

♦Weeks Worked in Follow-lJp Period 8 

Titie III 

Entered Employment Rate 64 % 

Average Wage at Pbcement * * * 



* New standards for PY1988-1989. 

♦* ^ P»Lploymcat and training program, 
p n I /^"ptiona! for governors. 
E.£jlrhl! ^s^d on those non-exempt. Rate rises 
to 50 % maximum in FY 1990. 



Figure 5 
Prospective Performance Standards 



KS FOOD STAMPS ^* 



BASIC OUTCOME MEASURES: Participation Rate 35% 

1. Individuals Placed 

2. Placement Transactions 

3. Obtained Employment 

4. Secured Employment (=1+3) 

EFHCIENCY MEASURES: 

5. Placements per Staff Year 

6. Placement Transacdons per Staff Year 

7. Obtained Employment per Staff Year 

8. Secured Employment per Staff Year 

WORK TEST MEASURES: 

9. UI Claimants Placed 

10. UI Claimants Obtained Employment 

1 1. UI Claimants Secured Employment 



results. For PY 1988-1989, governors are being encouraged to pursue longer-term, more 
costly interventions where appropriate. Between-state vririation is substantial. For 
example, Massachusetts, which is in the midst of a long-running economic expansion, has 
opted to assign only minimal importance to cost standards in recent years. 

The PY 1988-1989 Title III standards are outcome-oriented as well. As has been 
the case for the past few years, an entered employment rate standard is required (for the 
states' formula-funded Title III programs only) with a national goal of 64 percent. 
Governors a^e encouraged :o set an optional average wage at placement standard. Of 
course, not having a clearly defined mission makes it difficult to determine whether these 
measures are appropriate proxies or not. 

Questions have been raised concerning the accuracy of various JTPA measures. 
Analysts have questioned the heavy reliance on short-term placement standards when the 
goal of the program is longer-term employment and earnings increases.^^ Most research 
has found placement (at termination) to be the best single measure of longer-term net 
impacts on earnings, giving placement-related standards a strong claim to validity .^^ With 
the addition of post>program sta .dards, the group of standards taken as a whole would 
seem to have sufficient validity, encompassing employment, earnings, welfare reduction 
and non -employment youth factors. The reliability of these measures is also cause for 
concern. The quality and consistency of JTPA data are less than desirable, at all levels; 
there is significant variation in the definition of such terms as participant, placement, etc. 
Unfortunately, how great the variation is remains unknown. 

ES, The Wagner-Peyser Act suggests both outcome and process standards, 
although only process measures - in-program efficiency standards - are required. All of 
the potential standards mentioned explicitly in Section /{hj -- placements, direct and 
indirect; wages at placement; and retention - are mer^sures focused upon outcomes . Both 
the placement measures and the wage at placement are relatively immediate indicators of 
success; placement constliutes a clear quantity measure and wages, a quality-of-placement 
measure. It is interesting that retention - a longer-term outcome measure - is suggested, 
in that such a measure has not generally been the focus of labor exchange discussions: the 
market is cleared and labor is exchanged, regardless of employment duration or retention. 
However, as some analysts have indicated, the better the job, the longer it is ^Ikely to last, 
and the more it is expected that other national goals will be attained, i.e., ducing the 
unemployment-inflation tradeoff and facilitating non-inflationary growth.^^ 

Of the eleven (11) standards offered for optional state and substate use, seven are 
outcom >tandards; the remainder are process standards (Figure 5).^^ ES has yet to 



^^For recent examples, sec: Lev ilan and Gallo( 1987), Orfield and Slessarev (1986). The debate has been 
going on much longer, however, first surfacing in the job training evaluation literature in a paper by Bonis 
(1980). 

^^See: Geraci (1984), ZomiLsky,et. al. (i ^^5), among others. 
49see: Holt, et. al. (1971). 

50dOL tag (1985), pp. 5-6. In pnor years, the efficiency measures were in terms of cost-per-placement. 



implement wage or retention measures, although some individual states have done so on 
their own initiative. Emphasis has been on the quantity rather than the quality of outcomes 
Some states have also implemented target group participation or placement standards, as 
well as standards for the number of individuals counseled, tested, and other measures. The 
split between efficiency and effectiveness measures is relatively straightforward: the 
efficiency measures, capturing the immediate results per staff year, are labeled as such. 
The rest are effectiveness measures geared to short-run, quantitative measurement. 

There are several additional issues, including one discussed at length in ES policy 
and research circles, the use of individual placement versus placement transactions 
measures.^i The labor market can potentially clear as many Limes as an individual enters it 
and can find an employer willing and able to hire him or her. The market does not care 
whether the individual returns once or a dozen times per year. Yet, many are 
uncomfortable with the notion that the mission of the ES is to serv',-e high-turnover (low- 
wage) labor markets. Another issue concerns the definition and rep jrting of "placements." 
This issue has increasing importance with the spread of individual and group job search 
approaches. In job search, job seeking skills are taught, either in individual, self-directed 
or group, peer-support se!' ngs.52 The question is whjther to count a job obtained by an 
individual acting on his own the same as one in which he or she was phced through a local 
ES staff referral or job development effort. The former is generally referred to as "obtained 
employment" or a "direct placement," the latter an "indirect placement" or simply a 
"placement." The same question can be raised for JTPA given the prevalence of short-term 
job search as an activity. 

Where JTPA has operated within a very flexible environment in terms of 
measurement, definitions. MxS systems, etc., ES has historically adhered to tight reporting 
along federally-prescribed guidelines. The Employment Services Automated Reporting 
System (ES ARS) forms the basis of much of the nation's ES reporting, despite the fact that 
It was officially dropped as the mandated system several years ago. This has resulted in 
relatively accurate and consistent measurement of ES participation and outcomes. Given 
the focus on the ES' labor exchange mission, it can be presumed that the measures in place 
meet both validity and reliability tests. 

Other Programs. In the remaining programs, the primary attention to 
performance standards is in terms of participation requirements , especially for WIN and 
Food Stamps. These two programs contain procedures for prescribing who must register 
for and/or participate in program services (mandatory registrants) and who may be 
exempted from doing so. WIN determines mandatory target groups - not levels - by 
exclusion (e.g., women with pre-school children). The Food Stamp program has 
established mandatory participation rates, gradually increasing each year, in addition to 
addressing participation by target groups. Neither program has outcome standards for state 
or local programs. It should be noted that state funding for AFDC is affected by the degree 
to which they maintain low AFDC payment error rates , a clear efficiency .standard , but not 

^^This is part of the discussion on "employment service potential." See: Hanna and Zcund (1979). 
^-'For a concise ireaiment of job clubs, see: Wegmann (1986). 
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one lied to the delivery of services to promote recipient self-sufficiency. Outcome or 
effectiveness standards for welfare-to-work programs have tciken a back seat to 
participation and efficiency (of payment) standards. While research has been conducted on 
what appropriate outcome measures and related implementation issues might be, results- 
oriented standards have not been instituted to date 53 There is little doubt presently that 
such standards could be designed and implemented for these progra^ns. It is likely that, 
barring the passage of national welfare reform legislation and greater conformity of state- 
operated programs (e.g., benefit levels and cov^^'age especially), outcome standards would 
be in the form of proxies with large state (and even local) variation. One study of possible 
JTPA welfare standards found that post-program participant earnings served as a good 
proxy for welfare dependency reductions, though it recommended that both the proxy and 
some direct measure of welfare status be utilized^"^ 

Voc Ed has given little attention to performance standards tied to results of any 
kind. As indicated earlier, there is considerable confusion over who should be served and 
what the mission of the entire program is and should be, making the design of appropriate 
measures of performance difficult if not impossible. For Voc Ed, the question is: 
appropriate measures of what ? Voc Ed, secondary as well as postsecondary, is in need of 
serious goal clarification, a problem complicated by the fact that the lion's share of funding 
is non-federal. The strings are tied tightly to a very small share of the funding available 
locally. Performance measures appropriate for consumer homemaking for non- 
vocationally-oriented individuals would not be appropriate for electronic assembly 
programs for unemployed household heads. In the federal Perkins Act portions, the 
attention is on access or participation by specified target populations, such as single 
household heads, those with limited English proficiency, the disabled, and others. With 
the program provisions regarding sex bias/stereotyping, there is an implicit need for 
equitable service and results for participating groups. Little has been done to effect the 
needed measurement, and without major change in the system or clarification of its 
mission, little is likely to be done. 

E. Target Groups 

These programs have overlapping target populations, some more so than others. 
Open eligibility is the rule for programs such as labor exchange, while others, including 
JTPA and welfare-related programs, have conditional eligibility and are targeted on 
subgroups defined by demographic characteristics, status or means. Moreover, within 
these programs, eligible subgroups are often identified for service priority. These 
provisions concerning who is to be served and with what priority suggest that measures 
and standards for certain target groups are important issues cutting across program lines. 

Target group issues are especially important in programs which are tightly targeted 
and which seek to increase access/participation and to improve the status of certain groups. 



^^Again, there has been considerable research on the topic, including: Kogan, ct. al. (1984); ^''iichcll, et. 
al. (1980); and NGA (February 1987). 
5^Kogan, et. al. (1984). 
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The nature of these measures -- level of detail, number of standards, etc. -- further depends 
on a number of factors, among them the level of use and the type of eligibility 
(open/conditional). Generally, there appear to be at least four distinct uses for performance 
standards in the context of target groups: 

• Increasing service levels for key groups; 

• Improving equity for key groups, in part by emphasizing services and 
outcomes for them; 

• Ensuring performance for key groups; and 

• Minimizing barriers to serving key groups, by factoring them into 
adjustment mechanisms for calculating performance standards or other 
means. 

Each is a different role which can be served by standards and incentives and represents, in 
part, the implementation of targeting or prioritization of key groups via the standards 
process. The brief discussion that follows moves from programs with relatively open to 
those with restricted eligiBility. 

ES. Of all the programs, ES has the most open eligibility: any jobseeker ~ 
whether employed, unemployed or not-in-the-labor force (e.g., in full-time scnool, caiing 
for a sick parent) - may utilize ES labor exchange services.55 However, several groups 
are either required to use the ES or have priority of service (Figure 6). The enabling 
legislation targets UI claimants, the handicapped , and other " groups with special needs " as 
defined by governors. One of the explicit ES activities is administering the UI 'work test' 
and providing job finding/placement ser\'ices for UI claimant." [Section 7(a)]. In addition, 
ES must promote and develop jobs and serve the handicapped [Section 8(d)]. Governors 
discretionary funds may also be used for services to "groups with special needs," through 
joint ES agreements with PiCs, other public agencies or private, non-profit organizations. 
Two other groups have received special emphasis under compliance standards issued by 
the federal USES: veterans and misrrant/seasonal farmwork ers. 56 

The USES, through a series of TAGs, has encouraged states to establish sepai-ate 
placement standards only for UI claimants .^? One study of state implemertation of 
standards under Wagner-Peyser for the three-year period ending with FY 1985 found that 
more than half of the states had instituted target group standards for UI claimants, roughly 



55Thc same applies to employers wiLh a few caveats. Employers who are in the midit of a strike or 
lockout may not use the ES to help replace ihcir workforce, and government (federal; contratior^* are required 
to list job openings with ES. 

5^In each case, the standards were a response lo a court order. 
57u.S. Department of Labor (1985). 
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Figure 6 
Major Program Target G 



JlEA 

Title IIA 

• Economically disadvan- 
taged adults & youth 
(Note: SDA - level 

40 percent youth 
expenditure requirement) 

• Welfare recipients, 
including WIN rcgristrants 

• School dropouts 

• Other "hard-u>scrve" 
individuals (defined by 
governors) 

Title III 

ni<;locatcd Workers , including: 

• Terminated or laid-offUI 
eligibles/exhaustces; 

• Plant closure terminees; or 

• Long-term unemployed with 
limited opportunities in 
same/similar occupation. 



Open eligibility , with 
special priority for. 

• Ui claimants; 

• Handicapped" and 

• "Groups widi special 
needs" (defined by 
governors) 

Special emphasis on: 

• Veterans; and 

• Migrants/seasonal 
farm workers 



VOC ED* 



Title IIA special 
populations , include: 

• Economically or eriuca- 
uonally disadvan'iaged, 
including limited English 
proficiency, 22 percent; 

• Adults in need of training 
retraining, 12 percent; 

• Handicapped, 10 percent 

• Single parenl$/ 
homemakers, 8.5 percent 

• OffendcfS, 1 percent 

Also, persons in programs 
for sex equity or training 
fornon-tradiuonal 
occupauons, 3.5 percent 
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Q * Posisccondary. 

I Employment and training programs. 



WIN 



t^OOD STAMPS ** 



Able-bodied AFDC 
recipients . 16+ years of age , 
must register, unless 
exempt, as follows: 

• Caretaker of children 6 
years of age or under, 

• III, incapacitated; 

• Over 65 years; 

• Caretaker of ill, 
incapacitated family member; 

• Working 30 or more hours 
per week; 

• Children aged: 

16-17 and full-time students; 
18 and graduating; 

• In third trimester of 
pregnancy; 

• In areas too remote from 
programs; or 

• In AFDC foster care. 



Physically & mentally fit 
re cipients, over 18 and unde r 
60 years of ^^e, must register 
unless exen.pt as follows: 

• WIN or UI work registrant; 

• Caretaker for child under 6 
years or incapacitated person; 

• Student enrolled at least half- 
time in school or training; 

• Regular participant in drug/ 
alcohol program; or 

• Employed 30+ hrs per week 
(or less if weekly earnings 
below 30 X Federal minimum 
wage). 

Note : Odier individual and 
categorical exemptions as 
defined by stales. 



a third for veterans, less than a fourth for handicapped, and under a fifth for youth.58 
Target group standards were reported for welfare recipients, minorities and other groups' as 
well. Given the shift in the WIN program during the 1980s towards WIN Demo progran.s 
with single agency (welfare) administration, it is not surprising that welfare recipients 
receive very Uttle targeting attention in standards. The emphasis in ES to date has been on 
ornggmg standnrd.s -- typically, placement - for the particular target groups, while the 
legislative provisions address primarily access, equity and service levels almost as much. 

Postsecondary Voc Ed. The 1984 Perkins Act reauthorized Voc; Ed as a very 
categorized program federally, prescribing cenain groups for services with specific funding 
allocations, but largely leaving the mix of activities and services to the discretion of the 
states and local providers. Voc Ed was designed both to assure access for cenain 
historically-underserved groups and to enhance the quality of the services provided and the 
outcomes resulting. Taken as a whole -- considering programs operated with federal, state 
and local funds -- Voc Ed at all levels is mostly an open system in terms of eligibility; it is 
the federal ponion, a small fraction of the total, which is targeted and categorized. The 
targeting provisions constitute an effon to leverage states and localities in the direction of 
serving the enumerated special groups. 

One of the Act's'purposes [Section 2(2)] is to "assure that individuals who are 
inadequately served under vocational education programs are .'.ssured access" to quality 
programs, especially referring to the disadvantaged (academically and economically), 
handicapped, single parents/homemakers and oihers (Figure 6). Title II (Basic State 
Grants) lunher mandates that 57 percent of the federal funds are for Part A, Vocational 
Education Opponunities, sometimes referred to as the Special Populations part. This part 
in effect ranks the imponai'ce of serving these groups by allocating a specific funding 
amount for each. In descending order of funding shares, the target groups are: 
disadvantaged, 22 percent; ad, Its in need of t-^ining, 12 percent; handicapped, 10 percent; 
single parents/homemakers, 8.5 percent; sex bias/stereotyping, 3.5 percent; and offenders, 
1 percent. Several points can be made about this prioritization. First, to be "academically 
disadvantaged," an individual must have scored in the lowest 25th percentile on a 
standardized test, attained only a 2.0 average (4.0 scale) in secondary school, or failed to 
attain minimal academic standards. With the economically disadvantaged component, this 
stUl constitutes relatively open eligibility. Second, the limited Englisu proficiency group is 
omitted from the Title IIA funding pots, although they may also be served under other 
parts. Finally, it does not seem appropriate to include as "special populations" men and 
women in nontraditional occupations and sex stereotyping. This "group" is better 
classified as a "program." 

With elaborate targeting and categorized programming and v. Ith program purposes 
including access and higher quality services and outcomes, standards, "if these are to be 
instituted, need to be moving towards measures of participation levels and outcomes for 
these groups. If services/access and outcomes are so important that the program federally 
has instituted strong targeting via funding setasides, then establishing both service level and 

58cook and Fisher (1986), p. 4-7. 




outcome standards seems to follow. Outcome-based standards could easily be set by 
program subpart and vary with the experience in serving the groups in each. 

One other significant hindrance applies to the definition of any of these groups. In 
Voc Ed, unlike most of the other programs, it is often unclear who is oTicially a "student." 
A substantial portion of those enrolling for postsc ^ndary Voc Ed do so intermittently, one 
course at a time, over a period of years. They may only become part of an official program 
with a sequence of courses, as their separate courses add up to s(^mething approximating a 
whole. In this context, who constitutes the target mcasureni.^nl group and what are the 
management implications at the state and local level of instituting target f,roup standards, 
process (access) or outcome? Clearly, if it is unclear who the students are, it is likely to be 
almost equally difficult to determine who the "graduates" or "completors" are. In fact, only 
at the point an individual files for graduation and lakes a degree or certificate will his or her 
status as a student may become cleai in many cases. This issue concerning the definition of 
students and completors does not ari^e in the same way in programs with more restricted 
eligibility. The activities and target groups, as well as a major thrust of Voc Ed's mission, 
overlap substantially with JTPA, WIN and other programs. 

JTPA. Since Title III of JTPA has no special targeting beyond its relatively 
ambiguous eligibility provisions, and since Title IIB is exclusively for youth, the focus 
here is on Title IIA which serves economically disadvantaged adults and youth. Several 
target groups have been identified legislatively and separate standards have been 
established. The Act differentiates between adult and youth services and suggests 
performance measure differences accordingly. Separate standards have been issued for 
adults and youth , as well as for v.elfare recipients . For PY 1988, the Department has 
instituted twelve standards for these groups, two of which are for cash welfare recipients. 
Separate standaids have been e^t^ blli^hed borh because of legislative mandates and because 
of acknowledged differences in the piogr am^ for and labor market behaviors of the groups. 

One of the mort troubling aspects of the differentiation concerns that for adults and 
youth. The real differences are found not t>o much between adults and youth, as between 
adults and Ot -of-school youth compared to in-school youth. Out-of-school youth, such as 
teen parents with parenting responsibilities, are far closer to adults in behavior and in 
choosing employment- oriented programs than they are to other (in-school) youth. It is 
difficult for standards to account for this difference, independent of the program activities 
and services provided. This differentiation does not arise in a significant way in the other 
programs although it is being addressed in a number of the welfare reform bills in terms of 
the expected programs and desired outcomes for teen parents lacking high school diplomas 
versus other welfare recipients. 

The Act also establishes SDA-level service requirements (quasi-pardcipation 
standards) for the following groups: youth . WIN retristrants (whether mandatory or 
voluntary) and .school dropouts (Section 203(b)]. At least 40 percent of an SDA's available 
funds must be expended on eligible >outh, defined as economically disadvantaged, 14-21 
year olds. In addition, WIN registrants and school dropouts must be served on an 
"equitable basis, taking into account their proportion of economically disadvantaged 
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persons sixteen years of age or over in the area." (Dropouts are defined as those neither 
attending school nor subject to compulsory attendance laws and who have not received 
high school diplomas or the equivalent.) Of these, the youth provisions have received the 
most attention. All three requirements have typically been dealt with as compliance items 
rather than participation standards, per se . 

Finally, the Act provides that performance incentive grant funds shall l»e used by 
governors to reward programs exceeding their standards "...including incentives for 
serving hard-to-serve individuals " [Section 202(b)(3), emphasis added]. This section 
allows governors to designate hard-to-serve individuals but gives no indication as to how 
to do so. States have taken very different directions in defining the hard-to-serve, 
including declaring entire groups hard-to-serve and awarding grants for increasing service 
levels to them after the fact, as well as providing before-the-fact project funding to induce 
increased services to certain groups. Some states have devised planning documents to lead 
SDA staff through the process of identifying barriers e.g., skill deficiencies, support 
system problems, disabilities - associated with service difficulty. National standaixls have 
not been developed, since this is clearly a governor's decision.59 

JTPA is a restricted-eligibility program, with special targeting on youth, welfare 
recipients and dropouts, many of v/hom are also targeted for service in Voc Ed and WIN 
especially. There is potentially room for group-specific participation standards, as well as 
for the types of outcome-based standards now in place. All four of th^ uses for targeted 
standards outlined at the start of this part arc vided in JTPA. Governors are required to 
ensure equity of service for . /tain groups ^jd have the discretion to reward SDAs for 
increased services to the hard-to-serve (defined as they decide), to use group-specific 
standards to emphasis outcomes for them and to maJ.e adjustments to existing models or 
those of their own design for calculating standards adjusted for group-specific differences. 
It is important that underlying assumptions and expectations be made explicit before 
establishing such standards. Like CETA before, JTPA has been ciiticized for serving only 
the "cream" of the eligible population.^O This is a more complex phenomenon than most 
policy analysts understand and requires detailed analysis of program goals, constraints and 
operating environments (e.g., lack of stipends, limits on support services, contracting 
practices), as well as the nature of the population interested, willing and able to participate. 

Welfare Programs. With the exception of UI claimants in ES programs, in 
non-welfare programs participation is generally a matter of individual choice given 
eligibility. In welfare-related programs, however, the issue is not so much eligibility for 
participation but which groups will be required to register/participate as a condition of 
continuing to receive assistance. For WIN, Food Stamps and other related programs, there 
are two categories of individuals participating or registering to participate in employment 
and training activities: mandatory and voluntary registrants. Classification into one group 
or the other tended to be a relatively simple matter prior to the changes in the early 1980s. 



59For a recent paper discussing the hard-to-scrve issue, sec: Barnow and Constantine (1988). 

^^For example, sec Levitan and Gallo (1988). For more balanced discussions, see: Barnow and 
Constantine (1988); and Sandell and Rupp (1988). 
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For WIN, it is important to note that states determine eligibility for AFDC. While 
benefits are generally available to single heads of households with children (and low 
incomes and few assets), about half of the states offer benefits to families with both parents 
present, the unemployed parent or AFDC-UP option. All able-bodied AFDC recipients 
who are 16 years of age or over must register for employment and training, unless exempt, 
or lose their benefit eligibility. The registration requirements are thus determined by 
exemption. By law, those not required t • register may volunteer to participate in the 
programs. The exempt groups are: parents/caretakers of children under 6 years of age; ill, 
incapacitated, under 16 or over 65 years of age; individuals needed to care for ill or 
incapacitated family members; those working 30 or more hours per week; children 16-17 
years of age and in school fulltime; children 18 years of age who will graduate before 
turning 19; those in the third trimester of pregnancy; those living in areas with programs 
too remote from their residence; and those receiving AFDC foster care. Parents with young 
children are by far the largest exempt group. At least eight states have opted to require 
women with children younger than 6 years of age to register for services.^^ 

The proportion of volunteers varies widely from state to state and even county to 
county. At one extreme, four states go so far as to operate "universal registration" for 
WIN, though this is largely a paper process only. About thirty percent ot the states report 
fewer than 10 percent voluntary participants, while four states report 30 percent volunteers, 
including Massachusetts at 35 percent. Three or 4 states are operating what are essentially 
voluntary WIN programs. (In Texas counties accounting for most of the AFDC caseload, 
the rauo of mandatory- to-voluntary registrants ranged from a low of almost 1.2 to a high of 
nearly 3.5; the ratio of mandatory-to-voluntary registrants tended to be greatest in very 
large, urban countief 62) Priority of service among groups required to participate is left to 
the states, although with reduced WIN funding, priority is often given tc those with fewer 
barriers to participation (e.g., transportation, child care). 

Recent research suggests interesting directions for group-specific standards for 
welfare programs. Researchers evaluating state welfare-to-work initiatives around the 
country have performed intensive research on programs (for mandatory registrants) in San 
Diego, Baltimore and selected counties in Virginia, focusing on the design of proxy 
measures for improving the net impacts of employment and training programs for AFDC 
recipients. Their preliminary findings indicate that "... unless subgroup differences are 
taken into account, current performance measures may be sending the wrong s gnals to 
program administrators about the groups who should be receiving priority for program 
services."63 They conclude that key targetin^^ variables are prior work and welfare history, 
and that measures used should be changed to encourage targeting on more dependent and 
less job-ready (WIN-mandaiory) AFDC recipients. 



6^For this and the following figures, sec: Nightengale and Burbridgc (1987), p. 88ff. 
62King and Schcxnaydcr (1987), Tabic 24. 
^^Friedlandcr and Long (1987), p. xii. 
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Major changes in eligibility, participation and performance standards would be 
enacted in the leading welfare reform bills [H.R. 1720 and S. 1511]. First, all states 
would be required to provide assistance to two-parent families. One parent would be 
required to participate in two-parent families; the second parent could also be required to do 
so as long as child care is provided and the youngest child is 3 years of age or over. In 
addition, both proposals would explicitly prioritize groups for service. If resources were 
limited, NETWork (H.R. 1720) would require service priority to [Section 416(c)-(d}]: 

1. Volunteers whose family meets at least two of the following: 

a. Teen parents/parents under age 18 when first child bom; 

b. Family continuously in receipt of AFDC for 2 or more years; 

c. Fannly with children under age 6 years. 

2. Mandatories whose family meets at least two of the above criteria. 

3. All other voluntary participants. 

4. All other mandatory participants. 

The JOBS program (S. 1511) would encourage targeting on the following, in no special 
order: 

• Those on AFDC for 30 consec^iti ve months , with a child 3 years of age or over; 

• Those on AFDC for 3 0 of 60 months , with a child 3 years of age or over; 

• Custodial dropout parents 21 or vouneen and 

• Parents in two-parent families eligible due to unemployment of the principal wage 
earner. 

While JOBS has relatively flexible provisions on establishing standards, NETWork's 
provisions are integrated tightly with the requirements for service priority, the latter even 
ranking the importance of the standards to be established. Most importar/ are standards 
measuring the degree to which states are targeting the identified priority groups and 
providing intensive services for them; last are standards setting placement rate expectations 
for these groups [Section 436(a)(l)3. There are conflicting signals, however, in that the 
sarne section states, "... performance shall be measured by outcome and not by levels of 
activity or participation Whether tied to participation, activities or outcomes, the 
standards envisioned in the NETWork proposal are closely related to the groups prioritized 
for participation. Targeting on these groups u firmly roo:ed in recent research findings. 
But, the standards for these groups are far more detailed than subgroup standards or 
measures contained in JTPA, ES or any other existing program.64 

Food Stamp programs are targeted on a narrowly-defined segment of the Food 
Stamp recipient population - only an estimated 1.9 million (10 percent) of the total 19.4 
million perr-.is per month actually receiving Food Stamps in FY1987, and the standards 



o^Note that the JTPA Performance Standards Advisory Commiuecs and Technical Work Groups have 
considered and even recommended adding some additional measures with supporting data items which 
closely parallel these priorities, including such components as length of prior AFDC receipt. 
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clearly need to reflect this.^5 Those reojnred to register for work under the new Food 
Stamp programs are basically the same as under previous programs. Like WIN, mentally 
and physically fit, adult Food Stamp recipients are generally subject to work registration 
requirements unless they fall into one of the exempt categories (shown in Figure 6). Note 
that some two-thirds of all Food Stamp recipients (almost 13 million per month) are 
exempted because they are either children, elderly or disabled. Of the remaining recipients, 
most are exempted because they are caretakers, WIN registrants or already employed full 
time 66 Each state then determines the categories of and individual work registrants it 
desires to exempt, subject to federal approval; those remaining constitute their population 
for service in the programs and i\t group subject to performance siandards.'-^'^ Food 
Stamp work registrants are about evenly divided between men and women, have children 
(60 percent), are single parents (50 percent), receive income from general assistance (20-25 
percent), ??d aie employed through not for enough hours or earnings to be exempt (about 
14 percent). Moreover, they are likely to be relatively short-term recipients, with median 
stays of about 3 months, much shorter than for WIN registrants. 

F* Selected Implementation Issues 

Many issues related to implementation have been examined earlier. Two merit 
separate attention: ilata or information issues; and issues related directly to the 
implementation of standards, including policies for adjusting standards, contracting 
processes, as well as rewards and sanctions policies. 

Data Issues: Definitions, Collection and Reporting, Without 
information to support management decisionmaking at whatever level, it is difficult to 
establish proper performance expectations for measuring accomplishments. One important 
issue concerns the use of standard'zed definitions. The degree to which programs u" the 
same or similar definitions for critical program elements - including definitions of 
participants, enrollment, activities, and performance - varies v/idely. Greater 
standardization in definitions typically accompanies a stronger federal role. Programs 
which have explicitly decentralized policy and program decisionmaking to the state or local 
level also generally allow broader discretion in defining terms. Wider inter- area variation 
within programs can be expected where the federal funding share is smaller. 

A parallel set of issues is associated with data collection ana reporting for 
performance management. There is a question concerning the existing data collection and 
reporting systems, the underlying management information <^ystems which support 
perfomiance standards. For both participant and financial information, is the information 
system configured to provide data which is acce' Ible at the federal, state and local level? 
How do these information system capabilities differ across programs? Policymakers 
designing performance management approaches commonly make the fatal (implicit) 



^^Ccnter on Budget and Policy Priorities (1986). pp. 26ff. 

^^Eslimaics based on unpublished Congressional Budget Office calculations. 

^''Scc: Center on Budget and Policy Priorities (1986). pp. I6ff. 
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assumption tliat the requisite data will simply be forthcoming. It is illustra'\e that, despite 

a relatively determined effort to implement the performance standards requirements of the 

1978 CETA Amendments -- wiih 100 percent federal funding and considerable cooperation 

ai all levels of government, it took more than 3 years and r.ullions of dollars to assess, 

structure and fortify the state and local data systems needed to support the implementation 

of credible performance standards.^S Data issues appear mundane to policymakers, yet 

they provide the essential underlying foundation for any attempt to measure and manage 
performance. ^9 

Implementation Mechanisms and Rehted Policies. Standard provisions 
and policies are one thing; standards in practice are another. The implementation 
mechanisms for standards can vary widely. Numerical standards and the poi.cies to effect 
them are generally established at the federal and state level. In several of these programs, 
in keeping with the federal/state nature of the system, states may adjust standards, in terms 
of their numerical value or the weights accorded them. Not surprisingly, for these 
programs which attempt to prepare individuals for participation in the labor market, 
establishing across-the-board, national performance expectations is clearly not feasible or 
desirable. Expectations concerning placement rates, wages and related outcomes (or 
participation levels) will_ va'7 widely by state and locai conditions, demographics, etc. 
Adjustment policies are a key dimension of tlie implementation process. 

However, the ultimate translation of standards into programmatic reality for the 
service provider takes place locally via the contract document . Increasingly, standards are 
communicated to service providers through performance-based contracts. To fully and 
conr>pletely implement standards, the contract document must contain aU of il pertinent 
numerical standards or targets. Consider the situation in which there are a handful of 
federal or state performance standards, yet local providers' contract work statemerus and 
associated points for payment only contain a single outcome without targeting. In effect, 
the number of standards has been reduced to one, and the orientation of the program 
narrowed to a single dimension applied to all eligible groups alike. The content of the 
contract document is critical for implementing standaids. 

Another important implementation dimension is the type of re wards and sanctions 
EolMes accompanying the 3tandardi. All sorts of standards are possible for these 
programs, but more often than not, those which are instituted lack the rewards and 
sanctions policies needed to give them real meaning, the can-ots and sticks for program 
administrators and managers . There is some value in merely publicizing the fact that some 
programs did and some did not do well relative to a standard: the need to belong, to be 
associated with the "good guys," will motivate some managers to perform better than they 
otherwise would have; for others, it may have little or no effect if the magnitude of their 
funding or their day-to-day operating environment is unaffected. As a rule, program 



^hcc: King and Gcraci (1983) for further discussion. 

^^Pctcrs and Waterman (1982) r >tc that their excellent companies devote very serious attcnuon to 
measurement and information qua' -.y and quality. This is part of the larger p<..ini concerning the fact that, 
in ihcsc companies, results measurement h one of the things management holds tightly. 
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results will be belter where implications - positive and negative - are tied explicitly to 
performance. This is to be expected in a society dominated by notions of market-based 
principles and the ^bottom line*. 

Each of the programs wh!;h has reached the point of actually instituting 
performance standards - JTPA, ES and Food Stamps — has addressed these issues, but 
with differing degrees of success. The rest of the program") have these issues in their 
future should serious performance management and standards become rerJity for them. 

JTPA. The JTPA performance management framework for Title IIA programs is 
remarkable for its clarity. The legislation establishes a hierarchical system in which the 
Secretary develops the standards, sets the parameters within which governors may vary 
standards locally, and rules on sanctions. The Secretary has the discretion to shape the 
nature of performance management in several ways. He or she may estimate the national 
performance levels and add productivity improvement adjustments on top of those 
estimates. In addition, in prescribing the parameters for the governors, he or she can 
determine a great deal of the management prerogative at the state level. 

With the advice of their state councils and typically considerable input from the 
PICs and SDAs, governors now may prescribe variations in SDA-level standards, based 
on differences in economic, geographic and demographic factors, the service population 
and services provided. In some states (e.g., Oregon), the governor has chosen ns4 to vary 
standards across SDAs, but to apply them uniformly. Like the Secretary, governors may 
make productivity adjustments to the standards established or even establish additional 
state-specific performance standards. Many have done so. Further, governors have major 
responsibility for giving overall direction to their programs, in part by weighting standards 
as part of the incentives and sanctions policies. For example, giving low weight to the cost 
variables (e.g., Massachusetts) creates greater local flexibility for pursuing high-intensity 
treatments, while according more weight to the entered employment and cost standards 
le'^ds emphasis to short-term, lov/er-cost interventions. Moreover, governors also 
determine to what extent rewards (incentives) will be provided for performance relative to 
predicted standards. They may award incentives based on truly outstanding performance 
or merely for just exceeding the predicted levels. On the other side, governors also 
determine the degree to which sanctions will be applied to under-performing SDAs, as well 
as the focus and structure of technical assistance efforts in their state. 

Locally, PICs and SDAs plan and operate job training programs within the context 
of state-established policies. As mentioned above, local PICs also cany the legislatively- 
mandated responsibility for establishing youth employment competencies which underpin 
youth standard.^. In states with policies affording flexibility ^jr negotiating local standards, 
SDAs may al io bargain with governors to vary their stauuards beyonc that predicted by the 
Department of Labor*s adjustment model.^^ Few states offer this option in practice. 




^OBcyond ihc Model (1985). 
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Duta Issues. In the early stages of JTPA implementation, the Secretary and the 
Office of Management and Budget (0MB) granted virtually free rein to the states and local 
programs in defining terms critical to measuring performance. The absence of federal 
direction o*. the definitions of such teims as "participant," "termination," "employment," 
and "competency attainment, * coupled with minimal state/SDA reporting requirements, 
raised serious questions about the quality of program data available. It naturally has led to 
concerns over the ability of those data to support accurate esti. ation of performance at the 
national level. Improvements were recently proposed, seeking standardization of key 
definitions and additions to the reporting system, including items for reading deficiency, 
long-term welfare status, and minimal work history. 

Problems have been greatest for reporting youth competency outcomes, but have 
also existed in the information systerTi more generally. As the system moves into full-scale 
postprogram reporting ~ with hundreds of different entities, both professional sur\'ey 
researchers and program staff, conducting follow-up surveys - the problems are likely to 
multiply rapidly. In October 1987, the Task Force on Performance Management, 
comprised of state, local and public/businfcas group repress., atives, recommended a 
national survey on the reliability of JTPA postprogram data.'^l 

Implementation Mechanisms. Two policies constitute the primary implementation 
mechanisms for performance management requirements nationally: DOL's performance 
standards policies; and incentives/sanctions policies. The latter includes the related policy 
on incentives for serving hard-to-serve groups. It is becoming increasingly clear that one 
of the major forces behind JTPA's performance orientation is not standards or incentives 
and sanctions pgr gg, but the use of performance-based contracts locally, including fixed- 
unit price contracts. In each instance, federal policies are further modified and interpreted 
by governors before being put into practice locally. Even in the w .ning years of CETA, 
job training programs were beginning to make use of perfonnance-hased contracts as an 
effective means of controlling service delivery and ensuring accountability among their 
providers. It is no accident that the emergence of performance-based contracting coincided 
with the institution of PICs as new actors on the job training scene in 1977-19/8 under 
Title Vn of that Act. 

So far, there appear to have been no structured efforts to examine the 
implementation mechanisms for JTPA performance management and how li.ey relate to 
similar mechanisms for similar programs for the same populations. Within the Department 
of Labor there has been divergent policymaking on performance standards/incentives and 
performance contracting emanating from different segments of the agency. On the one 
hand, standards dxid incentives policies have increasingly sought to foster more intensive, 
higher cost treatments for remediation and training; on the other hand, other policies appear 
to be tightening the rules for contracting and constraining payments rela ".d to non- 
placement outcomes. Th^ has created unnecessary confusion in the day-to-day operating 
environment for job training. Another recent example of the difficulties which can arise 
with multiple agency policymaking and program offerings for such target populations is the 



7lTask Force (1987), p. 4. 
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implementation of the Greater Avenues for INdependence (GAIN) program in California. 
GAIN has had to address multiple program rules and regulations for JTPA, ES, welfare 
and Voc Ed, as well as the constraints presented by a highly autonomous and varied 
county-based delivery system. At last count, as many as five separate state-level agencies 
were issuing sometimes-conflicting interpretations of ilie operating inles for their portion of 
the GAIN program. 

ES. The system for establishing performance standards and awarding incentives 
within the federal/state ES is voluntary at the state and substate level, so much so that state 
participation in the ESARS data reponing system has been made optional. Though many, 
if not most, states still basically rely on ESARS-type reporting, this iS indicative of the 
chanf s in the labor exchange system in recent years. In terms of the definition of key 
terms and the actual coUecnon and reponing of ES performance, little has changed. Most 
states if asked will still turn to the definitions and forms used for so long, well before the 
shift in policy of the early 1980s. Basically, this means that there is no real reason to 
question the validity of the underlying data or its meaning. 

Two key ch ^nges have affected the ES in terms of implementation of standards and 
incentives. First, although there is little documentation to support the claim, it is evident 
that ES nationwide - as among the larger JTPA service providers - is being affected 
significantly by the spread of performance-based contracting . This is not to suggest that 
state/local ES operational relationships are being modified towards performance 
contravting, but that its financing and day-to-day operations are increasingly governed by 
such contracts as it does business with JTPA. ES continues to be a major JTPA provider. 

Second, the incorporation of Section 7(b) into the Wagner-Peyser Act allowed and 
encouraged much greater use of performance management approaches in ES, especially at 
the state l^^'d. As mentioned earlier, the Department of Labor's stance on ES standards, 
given th» ^ lative authority, has been noticeably passive. The experience with these new 
provisions to date has probably been as expecied; it has not been well documenied or 
publicized. Key findings from a recent nationwide survey of the implementation of ES 
performance standards and incentives policies include the following:^^ 

By PY1985, nearly all states were setting substate ES standards , although 
most of these used methodologies in lieu of or in addition to the Department* 
methodology. (Most did so due to the fact that the Department's mode 
underpredicted their performance historically.) The methods used varied widely. 

• The types of standard^; established also varied widely, though four out of 
five set standards for individuals placed ; more than half set placement transactions 
standards; and about a third set standards for obtained employment. Very few set 
standards for services provided. 



^^Cook and Fisher (1986). 



T^ggt group sta-^'Jards were established by about half the states, most often 
for UI claimants, but also for veterans, handicapped individuals and youth. 

Interestingly, while four out of five states used the new 7(b) flexibility to 
support projects for groups with special needs -- including efforts for youth, 
handicapped, UI claimants, older and dislocated workers -- only a little more than 
one in three was using these funds for substate performance incentives . (Increased 
staff was the typical form for incentives awards.) 

It is not at all clear that ES operations in most states have been altered significantly 
following the 1982 JTPA amendments to Wagner-Peyser. The voluntary nature of the 
standards and incentives policies, the prevailing passive federal policy, and the inherent 
tendency of these systems to change gradually has minimized the shift to an increased 
performance orientation. The interaction of ES with JTPA -- in joint planning, PIC and 
state council oversight, and performance contracting ~ at the state and local levels may 
produce more of a results outlook over time. Certainly the tools are present and the data 
appear to be valid and relatively consistent for those states that choose to go in this 
Jircction. 

Food Stamps. Food Stamp employment and training efforts are federal 
programs operated via state welfare agencies. Data issues which might trouble some other 
programs are thus not relevant for Food Stamps. In addition, the degree of federal control 
over the implementation of performance standards - again, as discussed, participation rate 
measures - is fully consistent with the funding sh^ „ although states have some discretion 
in defining »he base of Food Stamp recipients against whom £he standards will be applied 
and in varying the standards established nationally. 

In implementing the legisbtive provisions, the Department of Agriculture h?s 
established a 35 percent participation rate in :he Food Stamp employment and training; 
programs for all states for FY1989, rising almost immediately to the maximum 50 percent 
participation allowed under the legislation in the first quarter of FY 1990. This rate is 
calculated based on both voluntary and mandatory participation from the base of those 
eligibles, with states allowed to determine group and individual exemptions.73 States may 
make a case for lower participation standards, but the intent of the federal policies appears 
to weigh heavily against doing so. States must make their case prospectively based on 
documentation that their programs will require greater levels of effort because of the 
components offered or the type of individuals served. It is interesting that there are very 
direct sanctions for underperformance -- inadequate rates of actual participation relative to 
the standard -- in the form of proportionate disallowances of their federal administrative 
funds share; however, there are no corresponding incentives to perform ahc-t the 
standard. The program's performance standards and sanctions provisions are largely 
geared to minimum compliance with participation in low-intensity treatments by playing on 
the states' concerns over losing federal funds to cover already-incurred administrative 



'^For a discussion of Food Stamp standards and related issues, sec. Center for Budget and Policy Prionues 
(i986),Chs.V, VlandlX. 
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expenses. There are no rewards for moving participants into unsubsidized employment in 
higher paying positions. Nor are there incentives for reaching for higher program 
participation rates. Th^. states are essentially placed in a position of implementing 
mandatory federal policies for a predominantly federally-funded program. 
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IV. SYNTHESIS AND RECOMMENDATIONS 



Given rhat most of these pr -ams have been around for a long time - some more 
than half a century -- and that they share many of the same goals, the diversity in their 
performance management systems is remarkable. This wide variation is likely a reflection 
of differences in the phase of development, the period in which they were instituted, the 
Congressional commiuees which shaped them, the strength of the lobbies and interest 
groups surrounding them, and f(u too many other factors to son out adequately. 

From the lengthy discussion of performance management, programs and cross- 
cutting issues in this paper, a number of major and minor issues emerge which should be 
considered by national and state policymakers alike. Pressures for improved public 
accountability, for management towards clear performance objectives, and for 
nonduplication or coordination are only going to increase in the immediate future. Not to 
mention that participants and employers will also be se^ Jng effective services. Together, 
these are all "customers" the programs were designed to serve. The following discussion 
offers a synthesis, focusing on several sets of issues: broad contextual issues; major cross- 
cutting uesign and implementation issues; and several minor issues. Most of the major 
issues have to do with broad system design, structure and implementation. Finally, a 
number of recommendations are presented. 

A. ISSUES SYNTHESIS 

1. Contextual Issues 

Performance management systems, and the standards and incentives which are an 
integral component of these systems, should be designed v/ith their operational and 
environmental context in mind Three broad contextual issues arc of particular interest. 
Resolving these contextual issues can be viewed as the pre-condition for the designing and 
implementing effective management systems for human resource programs. 

Grant-in-Aid Programs. All of the programs examined here are federal grant- 
jn-^id programs. Although the extent to which they are funded from federal, state or local 
dollars differs widely, each is a grant-in-aid program with a substantial share of federal 
resources involved. Only postsecondaiy vocational education is dominated by state (and 
local) rather than federal funding. Historically, the education function generally has been 
more state and local than federal in nature. 

The programs' grant-in-aid character has imponant implications for designing and 
implementing management systems and measures. The following are especially 
noteworthy: 

Responsibility follows funding. The neater the share of federal funds, the 
greater the responsibility and the more federa strings can be attached to state and 
local programs. By this rule, the potential for federal direction is greatest in JTPA, 
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least in Voc Ed, with the others somewhere in between. Of course, th act that 
strings could be attached does not mean they have been. 

Shared fund'ng means shared governance . Grant-in-aid programs entail 
shared governance, with different levels of government having resprnsibility for 
certain functions. There is an inherent tension between federal ..ate and local 
actors in the pi ?cess of sharing the governing of any such program. 

• I nfluence, not control applies . Influence is the prevailing strategy in grant- 
in-aid programs, despite the fact that control is the standard term for the 
management function which standards, incentives and sanctions support. It is 
difficult to actually control the behavior of another level of government further 
along in the chain, whether federal-state or state-lo..al relationships. This applies to 
local entity-service provider relationships as well, although increasingly these are 
effected through performance contracts, resulting in enhanced control. Clt, ''ly, 
those programs with integrated state-local program delivery systems, sr::h as ES, 
WIN and Food Stamps, have greater potential for state-local control; JTPA and Voc 
Ed must rely more upon influence due to the relative autonomy of their local 
delivery systems. 

• Commitment is crucial at all levels . Unless state and local actors, providers 
included, desire to go in a given direction or can at least be convinced to want to be 
in the general vicinity, all the so-called controls conceivable — incentives and 
sanctions, too - will prove ineffective. Commitment to the desired ends, for the 
specified target groups, is necessary. 

In designing or redesigning performance management systems for these programs, 
these features suggest that more realistic, program-specific expectations are needed. The 
extent to which management systems with effective standards can be mandated and 
implemented from the federal down to the local level depends very much on the share of 
federal funding involved and the degree of influence which Cui> e cultivated . 

Enhanced State Discretion. The trend in the 1980s has been towards an 
enhanced state role in planning and operational decisionmaking for human resource 
development. This is especially evident in JTPA, ES and WIN. While the Perkins Act 
offers a counterpoint to this trend, states still retain enormous discretion in Voc Ed when all 
non-f jderai resources in that program area are considered; it is difficult, if not virtually 
impossible, to induce changes in state and local behavior in Voc Ed - in terms of services, 
tarf ^t populations or results — when such a small portion of the funds are federal and 
decisions have historically been made by autonomous local entitier. 

Governors have been vested with the responsibility, the authority and the funding 
to make the critical decisions regarding their human resources, and, increasingly, this 
extends to performance management functions. Two major constraints face the governors 
in managing these programs effectively: first, as elected officials, they may not be very 
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well suited for it; and second, the tool kits available to them may have been too tightly 
circumscribed. Both constraints are elaborated ^':iow. 

Market Bias. As performance manajem'ent has been introduced into human 
resource programs, there has been a tendency to search for market-based Mrincip''^3 for 
guidance. This has come hand-in-hand ^A^ith the establishment of PICs in JTPA and ES 
and the mover^ent towards greater private sector representation on the various state 
advisory councils. While underway for some time, emphasis on "bottom-line" 
management approaches in JTPA, ES, Food Stamps and now welfare reform has 
seemingly become an obsession. 

There appears to be near universal acceptance of the notion that merely allowing 
program operators to focus on services to major target groups in need is insufficient. 
Accountability or results-oriented management in the use of public funds must now be part 
of any and all programs. However, what must be acknowledged is that these are the very 
people and programs for which the market has apparently failed. To expect pure private 
sector or market-based approaches to adapt well is unreasonable. Applying out-of-context 
performance expectations and principles to such programs should be expected to lead to 
some of the phenomena which are now evident; namely, "creaming" or attempting to "beat 
the numbers" in JTPA as just one example. Not all "creaming" behavior in JTPA can or 
should be attributed to the application of standards, however, much can be ascribed to the 
combined effect of performance standards, PIC involvement, and the rise of performance 
contracting, as well as legislative constraints on the activities and support services which 
can now be provided. Before similar approaches are extended to Voc Ed or welfare-to- 
work programs, this apparent market bias merits careful scn^tinv . 

If private sector management principles and expectations are going to be applied 
successfully tc publicly-funded human source programs, modifications must be made. 
"Excellent" firms share several key attributes which distinguish them from their peers, one 
of which is that they possess simultaneous loose-tight properties. That is, the values of the 
organization — quality products, etc. -- and their measure .nent are held tightly, while the 
manner in which results are attained consistent with these values is left to the day-to-day 
operators. In human resource programs, ii appears that expectations are increasingly being 
elevated and that management increasingly mirrors the private sector. Yet, the ability of 
states and localities to do what is necessary to deliver is seriously constrained. One of the 
most striking examples of this phenomenon is that, while governors and PICs locally are 
under considerable pressure to perform with hard-to-serve groups such as welfare 
recipients, the ability to support continued participation by these very groups with 
allowances and to provide activities proven effective, i.e., public service jobs or work 
experience, has been severely restricted. This too needs a careful and realistic appraisal. 

2. Major Issues 

Ambiguous Goals. Many of these programs share both mission and goals. 
There is nothing inherently wrong with JTPA and the ES or JTPA and WIN seeking to 
move some of the same groups of individuals into jobs. A Utile competition goes a long 
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way in the public as well as ihe private sector. However, it may not be the best use of 
scarce resources, if carried too far. The mission of many of these programs (e.g., JTPA 
and ^ATIN) is moving individuals and their families towards self-sufficiency. But, there is 
enormous ambiguity in the actual goals of these same efforts. The worst offender in this 
regard is Voc Ed which appears to be a program for all reasons: Voc Ed exists to provide 
vocational and non-vocational (consumer economics/homemaking skills) training for 
"students" who plan to be both in and out of the labor market. It is all but impossible to 
hold such a system accountable or *o design an effective management system when its 
goals and objectives (and target populations) cannot be clearly articulated. 

While this is classified a major issue, it is hardly intractable. Most of these 
programs have relatively clear missions and may need minor clarification of goals and 
objectives. JTPA Title IIA has gone the farthest in this regard, laying out a clear statement 
of purpose and suggesting a number of measures, including immediate termination-based 
measures and longer-term postprogram ones, tied to program objectives and differentiated 
at least by adult and youth groups. It is hard to imagine clearer language than that found in 
JTPA. ES could use improved definition, and the welfare-related programs, as suggested 
by the reform proposals, appear to be headed in the right direction. Voc Ed alone seems to 
be left in a muddled state of affairs for any number of reasons, including not only its 
ambiguous mission and goals, but its funding and structure as well. Clarifying the 
programs' mission, goals anl objectives is the necessary first step. (Given the 
predominance of state nAd local Voc Ed funding, clarity will have to come both from the 
states as well as the Congress.) Tradition has it that if you don't know where you are 
going, all roads will lead you there. That may be, but probably not effectively or 
efficiently. 

Appropriate Actors/Level of Use, One of the Iirger issues arising from this 
discussion concerns the level of use, or the appropriate entity for use ' " e standards. 
Traditionally, responsibility for human resource developn:ent - from edui^^u^n to welfare - 
- has rested with the state.., and in the 1980s, increasingly the responsibility, authority and 
funding/financing for most related human resource efforts has been lodged at that level. 
The shifting of JTPA and welfare-to-work program responsibilities and the augmentation 
of the governors* role in ES bear this out. The logic behind this movement is sound to an 
extent: with governors responsible for programs serving similar populations and sharing 
relatively common missions, cost-effective, nonduplicative program performance in a state 
should result. But, there are real barriers to vesting performance management functions 
with governors, both with the systems as currently structured and with those now on the 
drawing board. 

First, there is the unavoidable conflict between distributional and results-oriented 
objectives. As elected officials, governors are more concerned with distribution than 
performance. In a coniest between who gets served and what is accomplished, the former 
is likely to win almost every time. For the governor as elected official - with an 
abbreviated time horizon and the difficulty of communicating the nuances of improved 
results at the margin to constituents - performance and distiiOution are probably one and 
the same. Good performance tends to be defined in terms of serving the right groups or 
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funding the riglu providers. In this regard, it is wofu. pointing out that even the bodies 
established to advise the governors on resource use and performance management, 
including state councils under JTPA, ES, WIN and Voc Ed, ve appointed by and serve at 
the pleasure of the elected officials. 

Trying the example of JTPA as currently structured, it is unclear why governors 
should take the process of ^anaging substate performance management at all seriously. 
Making full use of the tools provided - performance Incentives and sanctions applied to 
standards weighted as they see fit with the advice of their state councils - governors can 
make numerous friends (awarding incentives) and enemies (sanction: applied or incentives 
denied), but have nothing of obvious substance to gain : the state's JTPA funding is 
unrelated to the governor's managerial vigor. Not surprisingly, the JTPA system, with 
help from the federal and the state actors, has declared most SDAs winners, sanctioned 
very few, and generally undermined the credibility of the system's performance. Its public 
interest groups have taken to handing out awards for all sorts of non-performance, making 
the credibility problem that much worse. Systems need winners and lots o^ "hoopla** 
surrounding good results, but the performance celebrated also should be v/ Jely perceived 
as real. In part, this may reflect the political need to avoid making enemies, valuing 
distribution over performance. 

The state level may well be the appropriate one for human resource management 
decisions within federally-established parameters. However, it seems clear that, where 
governors have nothing to gain from carrying out distasteful management decisions, they 
are unlikely to do so with anything more than minimal compliance behavior. JTPA and ES 
experience to date is indicative. Moreover, if states could benefit from management 
decisions, there is still a real question as to whether doing so would be worth the cost 
politically. 

There are no clear solutions to this sticky issue, although there are options worth 
exploring. Granted all of the differences in size, complexity and cultural tradition, in 
Sweden, autonomous labor market boards are vested with similar performance 
management responsibilities. There is even precedent in this country where the resource in 
question is important enough: the Federal Reserve Board was created to regulate (national 
and state) money and banking matters. An apolitical, quasi-autonomous state entity ma;, be 
needed at least to oversee management performance of these varied human resource 
programs. (Several states, including Massachusetts, Michigan and New Jersey, have 
begun to experiment with such approaches.) It is di^'ficult to envision governors doing so, 
despite encouraging signs from them in attempting v/elfare-related initiatives. Note that it is 
much easier to launcn initiatives and to try new things than to sanction other elected 
officials and program providers or to reallocate funding for existing programs. Providing 
the governors financial incentives may be at least part of the ans . er. 

Appropriate Measures. Generally speaking, identifying appropriate measures 
of performance becomes a relatively minor technical issue once the more important 
decisions have been made about program mission, goals, objectives and target groups. 
This is not to minimize the technical difficulties involved, but to suggest that they are 
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feasible as long as the other decisions have been made. As indicated belovv, there appears 
to be little disagreement concerning the role which standards should play in program 
management, basically guiding policymakers and managers to more efficient and effective 
program operations. It is expected that this requires process and outcome, efficiency ar.d 
effectiveness standards. There is an important is5 ^ concerning the appropriate 
combinations of measures, one integrally connected to the target group issue. 

These programs have either open or restricted eligibility. ES and Voc Ed have open 
or relatively open eligibility for service, while the other programs all have some restrictions 
on who can be served. In addition, even within the more open systems, there are groups 
targeted for service priority or participation in special programs. Programs with open 
eligibility do not have the same needs for such process-oriented measures as participation 
rate standards. Instead, they require standards which will reinforce cost-effective service 
delivery to all comers. This will entail combinations of standards heavily oriented towards 
outcomes with both effectiveness and efficiency measures. ES standards, despite the fact 
that they are largely for optional use and that they carry some UI tai^eting, illustrate this 
case. 

On the other hand, JTPA and welfare-related efforts with tight eligibility and 
targeting will require combinations heavier on target group measures, including 
participation (e.g., WIN and Food Stamps) as well as outcome standards. The exact mix 
of standards varies with the particular program context, but the need for some standards - 
with the exception of the JTPA Title III program which is neces.sarily an ad hoc effort - 
clearly crosses program line^. It is also important to point out that, with the potential 
overlap beiween the systems and their target populations, if these measures are designed 
deliberately with attention to terms and definitions, operating cycles and system actors, 
among other program features - the use of consistent standards and incentives policies has 
the potential for rationalizing the delivery of services across programs in a given area, 
thereby improving overall program effectiveness and reducing unnecessary service 
duplication. 

Target Groups. Without belaboring the point, certainly for those programs with 
restricted eligibility and specified target groups, standards need to be differentiated by target 
group. This applies to both participation and outcome standards. JTPA Title IIA standards 
are very clearly differentiated by age, into adult and youth standards, based largely on the 
fact that programs for these groups have very different objectives and components. The 
extent to which standards need to be tailored to in- and out-of-school youth has been 
debated as well, as have the options for giving more direction to states on use of the "hard- 
to-serve" provisions which encourage program targeting via incentives.'^'^ Under the 
Perkins Aci, Voc Ed has shifted much more to a categorized system vith special 
populations targeted for service with the federallv-funded portions. It remains to be seen 
how effective this tail-wagging- Mie-dog approach will be without conforming provisions in 
the accompanying state-funded programs. (Another issue facing that system is coming to 
grips with seemingly simple terms and definitions, such ?is "student", "enrollment". 



^^Soo: Bamow and Constaniine (1988). 
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"termination" and other key items. It is hard to imagine effective target group standards 
without better uefinition of these items.) An effective case has recently been made for 
implementing measures differentiated by target group - based upon characteristics not 
ewisioned before - in welfare programs: prior work history and .\FDC experience.^^ 

Basically, the case being made here is two-fold: first, in programs geared to 
promoting access or equity of service for certain target populations, participation (and 
corresponding outcome) standards are appropriate; and, second, available findings from the 
evaluation research suggests that program effectiveness will be better promoted if outcomes 
and standards tied to them are explicitly differentiated by target group as well. It is worth 
pointing out that to a large extent, the JTPA "creaming" issue would be resolved with a 
combinatioi of target group-specific participation standards and outcome standards for 
those same groups. However, such strict controls on target group and outcomes might be 
viewed as inconsistent with the trend toward greater decentralization in grant-in-aid 
programs during the 1980s. After all, in al! of the debate on this topic, no one seems to be 
arguing that JTPA has been serving large numbers of ineligibles, just thai the eligibles 
seized were not as hard-to-serve or disadvantaged as oiliers in the population. 

3. Minor Issues 

Role of Standards. Standards should translate program goals and objectives 
into day-to-day reality and provide program operators the measures necessary for 
determining whether they are being accomplished on a timely basis. This applies whether 
the standards are tied to process or outcomes. There are few difficult issues remaining in 
terms of the role standards should p!ay. JTPA has gone a long way to defining this role in 
the Ti.. . IIA program, although there is some ambivalence between stPtcuients from high- 
level, national policymakers on their intent and the actual implementation into standards and 
incentives policies provided to states and localities: bein? a highly-performance driven 
system bftnt on longer-term investment is not fully consistent with standards based upon a 
projected 75 percent rate of SDAs exceeding those standards on the average The 
remaining systems, including ES, need to come to grips with this issue in ti.e near future if 
the ; Jtential for coordinated urvice delivery among programs serving similar target 
populations is to be realized. From the recent debates on welfare reform and earlier 
attempts to enact performance standards for Voc Ed, there is every reason to think this is 
beginning to happen. 

Adjustmen ^rocesses. Virtually all of the programs which have addressed the 
establishment of performance standards, whether participation or outcome, have provided 
for adjustment processes. Job training was the first in this regard, providing for 
adjustments to standards, based on the mix of clients, programs, local conditions, etc., a 
decade ago in the CETA legislation. In recent years, the JTPA system has tackled this one 
issue energetically, making use of considerable technical resources. The result has been the 
DOL Performance Standards Adjustment Model used by most states in the nation to adjust 
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SDA-level standards for Title IIA programs^^ More recently, some areas of the couritry 
have embarked on the development and implementation of alternative adjustment models, 
some based on participant-level data ' the:r ov^n states only.'' ES has taken similar steps 
but less actively. Food Stamp programs have explicitly incorporated modifications to 
required participation standards, while the other programs have only considered the 
prospects informally. There are provisions in some of the proposed welfare reform 
legislation which closely parallel JTPA provisions, particularly H.R. 1720. 

The issue which merits attention here has three pans: first, should adjustments be 
made; second, to what degree should adjustments be made; and, finally, what is the 
preferred mechanism for adjustment? It is not clear that the use of adjustments for any of 
these programs needs to be carried quite as far as it has been to date, panicularly if there is 
going to be a continuing involvement by and reliance on private sector representatives and 
volunteers. Both the JTPA and ES systems - the ones in use the longest have giown so 
complex and been modified so often that the adjustment processes have remained beyond 
the grasp of most and nearly unintelligible to the average person in contact with the system. 
Primarily, the process has been reduced to technocrat-to-technocrat communication with the 
rest simply asking for a report on the damage done. This adversely affects the credibility of 
the standards. Moreover, to the extent that the private s^;ctor is involved in oversight and 
responsible for performance via the PICs, there ^-eems to be some difficulty relating to 
adjustments for so many different factors and condiiions. In the private sector, more often 
than not, the producers are rewarded for overcoming adverse conditions, performing 
despite bad situations. The starting line or the length of the race is rarely altered to Rt the 
contestant as in the public programs. These are open issues, possible more appropriate to 
the discussion of private-public context differences. Yet, in this country, there appears to 
be almost an inherent competitiveness in sectors which drives PICs, local ES and 
welfare offices - and governors as well - to match their numbers against others*. 

B. RECOMMENDATIONS 

Of all the programs, JTPA has far and away the mc * coherently designed and 
implemented performance management system. This is true despite tht> struggles over 
control between federal, state and local actors, attempts to forge new and different roles and 
relationships between the public and private sectors, and the enormous changes which the 
progratr'' have been through in a veiy short time period. The JTPA system exhibits 
considerable coherence from goals at the front to performance measures at the back, from 
national-level Congressional/Executive oversight to local operation^, from the conduct of 
research and evaluation to the application of the key findings. In that system, there may 
even be a little too much of a good thing. There has been a clear and unchallenged 
dominance by economists and statisticians, quantifying and modeling the inputs and 
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outcomes of the job training system, fully embracing what others have referred to as the 
"rational model."' * 

Still, for ni lerous criticisms, it does seem to be working. It also seems to be in 
need of some mid-course corrections and a shakedown cruise by others less wedded to its 
development. Such corrections might include: giving the states' governors some real 
incentive for taking performance management seriously, like incentive awards at the state 
level; reducing substantially the number of standards in use; mainiaining the same standards 
in place for a 2-year planning cycle; developing ani adhering to a far simpler a^Jjustment 
model, one that would not require five years to explain well to PICs with frequen^ turnover- 
implementing mechanisms to more effectively promote service to the hard-to-serve; and 
others. Note that, all of these corrections 'suggestions aside, the system of job training has 
undeniably shifted from a process- to a performance -driven mode in a matter of a fe w short 
years. Refinements are in order; not wholesale restructuring. JTPA's per.ormance 
management system serves as the "standard" against which the others are assessed for 
better or worse. It is the only one which has been design^^J, implemented and tested. 

1 • Program-Specific Recommendations 

There are some program-specific recommendations to offer before turning to the 
broader set. Regarding the JTPA Title 111 Dislocated Workers Program , there appears to be 
no reason to proceed with what ir* largely a state/local charade regarding legislatively- 
mandated performance standards for several reasons. First, there is no compelling reason 
to have them or to use them. Displaced worker programs ar^. inherently ad hoc, temporary 
efforts to assist in situations including plant shutdowns and mass layoffs. Performance 
against standards and incentives/sanctions tied to them carry little weight when the 
programs will only be in place a short time under special, very adverse circumstances. 
Second, as constituted, the goals and objectives of the program are so poorly articu^i^ied 
that it would be difficult to design them. And, third, no data base currently exists, given 
state-level repor»ng, to devise *hem and which would support substale adjustments. It 
would appear preferable to substitute legislative provisions similar to those used in earlier 
decades requiring or encouraging funding of providers of "demonstrated effectiveness" 
only. 

Establishing performance s> stems for postsecondar^ Voc Ed programs which 
would be compatible with the other human resource efforts also iooks nearly intractable 
under the cuixent configuration. This is unfortunate, particularly in il^nt of the large and 
expanding role postsecondary institutions play in JTPA, WIN and other programs. 
Counting all federal, state and local dollars involved, the public as a whole has far more at 
stake in Voc Ed than in the other self-sufficiency programs.'^^ There is very little 
accountability incorporated into Voc Ed at any level, and, given the dominance of state and 



'^Spcicrs and Waterman (1982), Chapter 2. 

^^In Texas, the rough orders of magnitude in fundir.^ in 1986 were as follows: Voc Ed had nearly four 
limes the funding in JTPA Title II A, JTPA had about Uiree times the funding of ES, and ES had five tim 
the funding in WIN. Voc Ed enjoyed nearly fifiy timt^ the funding for the state's WIN program! 
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local funding, it is difficult to see federally-mandated changes making the difference. This 
is an issue which merits careful consideration in tlie upcoming Perkins Act reauthorization. 

2* General Recommendations 

The importance of human resources has never been greater. U.S. international 
competitiveness Innges more on the development of human resources than on policies 
affecting physical capital, a fact which has been taken to heart by the country's major 
trading partners. Successful human resource poliv "es, emailing far greater attention to 
program effectiveness and efficiency, are critical. The more general program 
recommendations regarding the design and implementation of effective performance 
management systems - including the management framework, the standards themselves, 
and related implementation policies - are offered below. 

The key federal function lies in shaping the basic direction of these programs and in 
creating and maintaining an appropriate playing field. From ih^ federal vantage point, 
there are several important recommendations, including: 

Quasi-autonomous state councils should be created with broad 
performance oversight responsibility for all of the human resource 
programs. In order to de-politicize the process of managing program performance, 
quasi-autonomous councils need to be created by federal legislation with broad oversight 
responsibility reaching across these programs. Membership on the councils would neexi to 
^•^ broadened to provide appropriate representation for all major groups affected. (Note 
that the governors already possess the authority to expand the existing JTPA State 
Councils' scope to encompass ES and WIN; these Councils also review the state Voc Ed 
and Food Stamp plans.) Although detailed options for implementation should be left to the 
states - including how they would be structured, the size of the staff, the length of 
members' terms, the appointment process, the number of meetings, etc. - several features 
are critical. Councils should: 

Have members appointed by governors, with terms which overlap and 
extend beyond that of the governor, so that no governor would possess 
complete control over the councils' makeup and decisions; 

Have a staff independent of the governor or any of the programs or 
agencies; 

Have oversight and evaluation responsibilities only; they should not operate 
programs or be involved in operational management decisionmaking; and 

Hold open meetin'^s and issue regular reports on program performance to 
the governor, the Siate legislature, the agencies and the responsible federal 
agencies. 
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The key functions of these councils would be to: 

Review program performance for all human resource pr ^grams operating 
with federal funds in the state; 

Advise the governor and the agencies on coordination and on establishing 
state-level performance standards for these programs, including processes 
appropriate for adjusting these standards at the sub-state level; 

Recommend incentives and sanctions policies for these programs to the 
governor and the agencies; and 

Prepare and issue regular reports on program performance, both within and 
across piv)grams. 

Thus, these state-level councils would provide :in independent, apolitical review of 
performance yielding an objective base of information which governors would use in 
managing states* human resource program performance. The response could certainly be 
political to some degree, but the information and analysis provided to elicit that response 
would be removed from the political proce. to the extent described here. This is at least 
the first step in the right dii'ection. 

• Incentives should be instituted to reward governors for good 
program performance at the state level Both ES and JTPA have some experience 
v/ith operating under performance-driven systems, but only at the state level below. 
WIN and Food Stamps have focused on non-outcomes measures of uerformance for the 
most part. It is important that, if governors are going to continue to shoulder performance 
management responsibilities for buman resource progiams, they have the incentives for 
doing s . . Just as sub-state entities under JTPA and ES receive the bulk of their annual 
funding based on need and a small but meaningful share (from 6-10 percent, roughly) 
based on performance, governors should operate under a similar re\^'ards structure. Th^^se 
governors who tr . performance management seriously — both rewarding good performers 
and sanctioning poor ones to induce better results over time - should reap the rewards for 
their states; those who do not, should not. The political process should take it from there, 
and appropriately so. 

The details of su an incentive structure encompassing the various programs are 
not obvious. Keeping any structure simple is an important criteria to keep in mind. At 
least for the separate programs, it would be possible to "roll up" the sub-state performance 
i.uo some state-level performance number, to tie the rewards to the number of standards 
exceeded, or many other approaches. The options are numerous, each with accompanying 
advantages and disadvantages needing careful examination. 

• There needs to be a general tightening of w_hg± is measured as 
program performance. For several years now, there has been a very strong sliift in 
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favor of greatly expanded state discretion, genenfily following the New Fede.-alism 
approach to social programming. Unfortunately, despite sizeable federal funding and the 
grant-in-aid nature of most of these programs, this shift has included greater laxity in 
defining and measuring the desired program results as well. As described earlier, in the 
private sector, tight definition and measurement of results is the rule, with accompanying 
flexibility in the means of attaining them for the high-performing companies. In most of 
these programs, there is very weak definition of the outcomes desired and almost no quality 
control ensure that reporting is accurate and consistent within, much less between, 
programs and states. Further, the JTPA practice of defining the nati :al departure points 
for standards at a level so that 75 percent are expected to exceed them is very discouraging. 
If the desired :etum tor expending federal monies is employment with earnings sufficient to 
support families above the poverty line, then does it make good policy sense to allow states 
10 defme placements (or enrollments) more loosely. States and local entities can always opt 
to define outcomes more tightly than the federal definitions prescribe, particularly via the 
performance contracting process. 

The corollary to this recommendation is that the lool kit available to the governors 
and their agencies may need to be augmenteu. It is decidedly ^onu^y to market-based 
principles of operation to tighten up performance expectations while handicapping the 
responsible parties. Governors and local programs have been given responsibility for 
meeting standards and local programs (in JTPA a^id ES) have been offered financial 
incentives for doing so, but they have also been handed a restricted set of progiam 
treatments and constrained in terms of th-^ amounts they can expend for administration and 
support services. This needs to be examined realistically with an eye towards balancing 
performance expectations and the actual operating environment. 

Programs should move toward utilization of both outcomes- and 
process-based performance standards. While recognizing that there are impor n 
differences between these programs, all at least share the goal of moving eligible 
participating families or their individual members into unsubsidized employment. Such 
outcome-based standards provide management the ^y- to-day tools for assessing 
performance, particularly where the measures implemented are proxies of the longer-term 
net mipacts desired. JTPA and ES have already implemented such measures; WIN and 
Food Stamps have yet to do so, focii^sing instead on payment error rates or on 
participation rate standards. {Nolt U A there is precede it in the old WIN allocation 
formulas.) In addition, it is increasingly clear from the research that outcome-based 
measures work best when disaggregated by target group. Thus, combining the two types 
of measures - at least for those programs with restricted eligibili.y or important targeting 
requirements - is desirable. 

Beyond these recommendations at the federal level, there are also several general 
ones which apply at the state level. The onus is on states - especially the governors - to 
take the responsibility for performance management in the current human resource proyam 
environment. In fact, for programs such as JTPA, ES and WIK .he onus has been there 
for some tirne though in varying degrees. Some states have taken the responsibilities quite 
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seriously, launching creative management as well as program initiatives.^^ Apparently, 
most have not been so active on the performance management side. With some of the 
recommendations listed above, there might be more reason to do so. 

Specific state-level recommendations are as follows: 

• Governors should take the Lad in giving clear, unambiguous 
direction to these prograr in their states and in defining the role which 
standards should play in getting them there. An issue which has surfaced for 
several of these efforts is the ambiguity of mission and goals. In the absence of national 
guidance on this count governors need to take the lead in clarifying the direction the 
programs are to take. Without clear missions and goals, it is not realistic to place 
substantial reliance on standards and performance management in getting there. Governors 
can and should play this role to the extent they can under their state structures. 

• States should take a comprehensive approach * rationalizing human 
resource service delivery, making use of performance standardc as a 
primary vehicle. Stages, especially the governors, need to take a more comprehensive 
approach to the delivery of human resource development programs, rationalizing de^'very 
of services across programs and among provider agencies, based solidly on performance 
against standards. Governors already have the leeway to carry out much of what is 
needed, but witli the rdded support provided by the state councils recommended above, far 
more could be done. The first steps might include initialing and managing the process of 
standardizing definitions for key program tenns, eliminating existing disincentives in state 
policies, etc. Governors could go a long way towards promoting effective and efficient 
service delivery equipped with objective information on performance in the various 
programs in their states. 

Finally, there ai'e a few general recommendations which should be considered. 
These recommendations are ;elated to the broader context within which performance 
management occms. These recommendations are as follows: 

• Efforts to design and implement performance management systems 
need to be fully cognizant of the programs' grant-in-aid nature. The sharing 
of governance among federal, state and local actors is dissimilar across program lines, 
much of which can be attributed to varying shares of federal funding. The same 
expectations for perfonnance management do not and cannot apply to ES with 100 percent 
federal funding and state-controlled local offices, and to postsecondary Voc Ed v^ith less 
than 10 percent federal funding and autonomous local entities. Expectations must conform 
to reality, not the other way around. 

^ Simpler adjustment models for performance standards need to be 
implemented, consistent with the preference for market-oriented 



^^Massachusetts, Michigan and JHorida come readily to mind here although there are certainly others. 
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approaches. It has become common in the existing programs (e.g., JTPA, ES and Food 
Stamps) to build in adjustment processes fo» performance standards which tuke in^- 
account any number of factors, such as local economic conditions, the mix of people 
served and program activities provided, etc. The proposed welfare reform bills outline 
similar processes. Yet, these types of adjustments are in conflict with more typical market- 
based approaches. In the private sector, firms or divisions witliin the firm are expected to 
surmount any obstacles in their path to make the desired profits. On the other hana, these 
same firms also have relatively unconstrained choices of inputs and production processes 
over time. Finns may and often do sell off non-pe*.. aning divisions to respond to market 
forces; if they do not, they find it difficult to raise funds in the financial markets. While 
this market bias may not be completely appropriate for human resource development 
programs which deal with the market's failures, it is unlikely to yield any time in the 
immediate future. This country is enamored with market-oriented processes even in the 
public sector. 

Models need to be developed which allow and ev^tn encourage the type of head-to- 
head competition which prev Jls in the private sector and which surfaces from time to time 
in these prDgrams as well. Such competition seems to be inherent in the systems and might 
not be detrimental as long as outcomes are adjusted to some degree for conditions outside 
management's control. Features of these simplified processes include the foUowmg, 
among others: 

Adjustments need to be few and intuitively sensible (^alid); they should be 
made where theic is a clear, empirical relationship to the outcome; 

Adjustments should be easy to understand and explain; and 

Adjustment mode* s should also remain relatively stable, changing little over 
time to impart credibility and a stable operating environment to t^ extent 
possible for the programs. 
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